Abstract:Increasing traffic demands, higher levels of automation, and communication enhancements provide novel design opportunities for future air traffic controllers (ATCs). This article presents a novel deep reinforcement learning (DRL) controller to aid conflict resolution for autonomous free flight. Although DRL has achieved important advancements in this field, the existing works pay little attention to the explainability and safety issues related to DRL controllers, particularly the safety under adversarial attacks. To address those two issues, we design a fully explainable DRL framework wherein we: 1) decompose the coupled Q value learning model into a safety-awareness and efficiency (reach the target) one; and 2) use information from surrounding intruders as inputs, eliminating the needs of central controllers. In our simulated experiments, we show that by decoupling the safety-awareness and efficiency, we can exceed performance on free flight control tasks while dramatically improving explainability on practical. In addition, the safety Q learning module provides rich information about the safety situation of environments. To study the safety under adversarial attacks, we additionally propose an adversarial attack strategy that can impose both safety-oriented and efficiency-oriented attacks. The adversarial aims to minimize safety/efficiency by only attacking the agent at a few time steps. In the experiments, our attack strategy increases as many collisions as the uniform attack (i.e., attacking at every time step) by only attacking the agent four times less often, which provide insights into the capabilities and restrictions of the DRL in future ATC designs. The source code is publicly available at <a class="link-external link-https" href="https://github.com/WLeiiiii/Gym-ATC-Attack-Project" rel="external noopener nofollow">this https URL</a>.

Tube-based robust reinforcement learning for autonomous maneuver decision for UCAVs

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

Trustworthy autonomous driving via defense-aware robust reinforcement learning against worst-case observational perturbations

Deep reinforcement learning and its application in autonomous fitting optimization for attack areas of UCAVs

Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm

Deep Reinforcement-Learning-Based Air-Combat-Maneuver Generation Framework

General multi-agent reinforcement learning integrating adaptive manoeuvre strategy for real-time multi-aircraft conflict resolution

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

Explainable and Safe Reinforcement Learning for Autonomous Air Mobility

Enhancing Air Traffic Control: A Transparent Deep Reinforcement Learning Framework for Autonomous Conflict Resolution

Autonomous UAV maneuvering decisions by refining opponent strategies

A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat

Robust Adaptive Ensemble Adversary Reinforcement Learning

Target tracking strategy using deep deterministic policy gradient

Multi-intent autonomous decision-making for air combat with deep reinforcement learning

Realizing asynchronous finite-time robust tracking control of switched flight vehicles by using nonfragile deep reinforcement learning

A Deep Reinforcement Learning Based Intelligent Decision Method for UCAV Air Combat

Efficient Deep Learning of Robust, Adaptive Policies using Tube MPC-Guided Data Augmentation

Memory-Enhanced Twin Delayed Deep Deterministic Policy Gradient (ME-TD3)-Based Unmanned Combat Aerial Vehicle Trajectory Planning for Avoiding Radar Detection Threats in Dynamic and Unknown Environments

An Intelligent Maneuver Decision-Making Approach for Air Combat Based on Deep Reinforcement Learning and Transformer Networks

Homotopy Based Reinforcement Learning with Maximum Entropy for Autonomous Air Combat