Abstract:A deep learning-based safety monitoring framework for air traffic control (ATC) systems is proposed in this paper to reduce human errors and relieve the controllers' workload by regulating the controlling procedure, eliminating communication misunderstanding, monitoring flight conformance, and detecting potential conflicts. The framework comprises automatic speech recognition (ASR), controlling intent inference (CII), and control safety monitoring (CSM) subsystems. The pipeline of the proposed framework can be described as follows: the ASR subsystem translates the pilot-controller voice communications (PCVCs) into texts, which are then converted to the predefined data structure by the CII subsystem. Three types of air traffic safety measures, including repetition check, flight conformance verification, and potential conflict detection, are finally validated by the CSM subsystem. An improved end-to-end ASR model with convolutional, bidirectional long short-term memory (BLSTM) and fully connected (FC) layers is trained using the connectionist temporal classification loss function. The BLSTM and FC combined CII model is designed to infer the controlling intent and slot filling. A language model is also trained in this subsystem to improve the overall performance of the framework. After converting the PCVCs to ATC data, the CSM subsystem checks the given safety monitoring tasks and sends warnings to the current system. The experimental results show that the proposed ASR model obtains a better performance than that of other approaches, and the tasks in the CII subsystem are fulfilled with a high classification precision. The CSM subsystem is also tested to confirm its safety monitoring function by playing back the data and several simulated instructions. To the best of our knowledge, this is pioneering work in the safety monitoring of flight control by recognizing the PCVCs with deep learning-based methods.

A Study on Real-Time Control Capability Assessment of Approach Controllers.

The performance assessment of multiple mode MIMO industrial process based on Data-driven controller

A Novel Non-Uniform Control Vector Parameterization Approach with Time Grid Refinement for Flight Level Tracking Optimal Control Problems.

A numerical evaluation of real-time workloads for ramp controller through optimization of multi-type feature combinations derived from eye tracker, respiratory, and fatigue patterns

An integrated framework for eye tracking-assisted task capability recognition of air traffic controllers with machine learning

Rapid Mental Workload Detection of Air Traffic Controllers with Three EEG Sensors

A Field Study on Safety Performance of Apron Controllers at a Large-Scale Airport Based on Digital Tower

Air Traffic Controller Workload Detection Based on EEG Signals

Fatigue Detection of Air Traffic Controllers Through Their Eye Movements

Research on assessment of air traffic control operation quality based on track data

A Real-Time ATC Safety Monitoring Framework Using a Deep Learning Approach

Air Traffic Control Command Monitoring System Based on Information Integration

Developing an Improved ACT-R Model for Pilot Situation Awareness Measurement

A Real-Time Measurement Model of Attention-Allocation Level and Its Application in Simulated SPO Task

A General Hierarchical Control System to Model ACC Systems: An Empirical Study

Human Factors Analysis of Air Traffic Safety Based on HFACS-BN Model

An Adaptive Airspace Sectorization Algorithm and Intelligent Controlling

Cognitive Workload Detection of Air Traffic Controllers Based on mRMR and Fewer EEG Channels

Reliability Prediction and Safety Evaluation of ATC Automation System

Airspace Sector Dividing Method to Balance Dynamic Control Workload

Assessing the Capacity of Air Traffic Control Secondary Surveillance Radar System