Projects

[Vibe Coding] Real-Time ATC Transcription for Pilots

Visualizing Air Traffic Communication in Real Time

ROLE

Designer/User

TOOLS

Figma, Claude Code

WHAT MAKES ME WANT TO START THIS PROJECT?

On February 25, 2025, a near-tragic runway incursion occurred at Chicago Midway Airport involving Flexjet Flight 560 (a Challenger 350) and Southwest Flight 2504 (a Boeing 737-800). Despite ATC’s instruction to “hold short” of Runway 31C, the Flexjet crew misheard and mistakenly crossed the active runway without clearance.

This serious communication breakdown between ATC and experienced pilots highlights how easily misunderstandings can escalate into dangerous situations. If such errors can occur among professionals, the risks are even greater for student pilots or less experienced aviators—making clear, accessible communication tools essential for aviation safety.

Overview

As both a licensed pilot and a non-native English speaker, I’ve personally experienced one of the biggest challenges in aviation: ATC (Air Traffic Control) communication. Fast speech, aviation-specific jargon, and the pressure of flying can make it difficult to catch every instruction. Missing or mishearing critical instructions can compromise situational awareness, safety, and pilot confidence.

This case study explores the design of a real-time ATC transcription tool—a system that listens to ATC transmissions and instantly provides text transcriptions for pilots in the cockpit.

The Problem

  • Pilots often struggle to fully understand ATC communications due to speed, accent, jargon, or nervousness.

  • In high workload environments, even experienced pilots can miss transmissions.

  • Current avionics systems provide no text backup—only audio.

  • A misheard instruction (e.g., wrong altitude or heading) can lead to dangerous outcomes.

HMW help pilots improve comprehension and confidence in ATC communications through accessible, real-time support?

The Million Dollar Question

Interviewing Pilots & Flight Instructors

  • Pilot Interviews (5 private pilots, 2 CFIs): to understand real-world ATC struggles.

  • Flight Observation: listening in on tower communications at a local Class C airport.

  • Competitive Benchmarking: looked at speech-to-text tools (Otter.ai, Microsoft Azure STT, LiveATC), but none were optimized for aviation.

Personas

Persona 1: Rodrigo

International Student Pilot

  • 28 years old, Spanish native speaker.

  • Struggles with fast ATC speech.

  • Goal: Increase confidence during solo flights.

  • Pain: Often needs to request “say again,” feels embarrassed.

Persona 2: Sarah

Private Pilot, Busy Airspace

  • 35 years old, English native speaker.

  • Goal: Reduce workload in busy Class B airspace.

  • Pain: Misses instructions when multitasking.

User Flow

Initial Sketches

Usability Testing in Simulated Flight Conditions

To evaluate the effectiveness of real-time transcription in a realistic environment, I conducted a usability test with four pilots during a simulator session. The prototype was deployed on an iPad, where pilots were asked to respond to ATC instructions with transcription support enabled. This setup allowed us to observe how the tool integrates into actual cockpit workflows under time-sensitive conditions.

Setup

  • 4 pilots participated in a simulator-based test

  • Prototype deployed on iPad

  • Task: Respond to ATC instructions using real-time transcription support

Improved Confidence & Faster Comprehension

Results

  • 100% of pilots reported increased confidence in understanding ATC instructions

  • Non-native English speakers noted significantly faster comprehension

  • Transcription helped “decode” complex or rapid instructions in real time

Actionable Feedback for Iteration

Feedback

  • Increase font size for better in-flight readability

  • Store at least the last 20 transmissions for quick reference

  • Add voice playback of the most recent instruction as a backup

Next Steps

This project explores how AI-powered transcription can better support pilots by integrating with existing aviation tools and improving accuracy in complex communication environments. Future directions focus on enhancing ecosystem compatibility, domain-specific intelligence, and real-world validation.

  • Explore integration with ForeFlight and Garmin Pilot to embed transcription directly into pilot workflows

  • Leverage AI-driven glossary identification to recognize aviation terminology and improve transcription accuracy

  • Validate performance in IFR scenarios and untowered airports where communication is dense and fast-paced

Projects