Abstract:With the development of unmanned aerial vehicle (UAV) and artificial intelligence (AI) technology, Intelligent UAV will be widely used in future autonomous aerial combat. Previous researches on autonomous aerial combat within visual range (WVR) have limitations due to simplifying assumptions, limited robustness, and ignoring sensor errors. In this paper, in order to consider the error of the aircraft sensors, we model the aerial combat WVR as a state-adversarial Markov decision process (SA-MDP), which introduce the small adversarial perturbations on state observations and these perturbations do not alter the environment directly, but can mislead the agent into making suboptimal decisions. Meanwhile, we propose a novel autonomous aerial combat maneuver strategy generation algorithm with high-performance and high-robustness based on state-adversarial deep deterministic policy gradient algorithm (SA-DDPG), which add a robustness regularizers related to an upper bound on performance loss at the actor-network. At the same time, a reward shaping method based on maximum entropy (MaxEnt) inverse reinforcement learning algorithm (IRL) is proposed to improve the aerial combat strategy generation algorithm’s efficiency. Finally, the efficiency of the aerial combat strategy generation algorithm and the performance and robustness of the resulting aerial combat strategy is verified by simulation experiments. Our main contributions are three-fold. First, to introduce the observation errors of UAV, we are modeling air combat as SA-MDP. Second, to make the strategy network of air combat maneuver more robust in the presence of observation errors, we introduce regularizers into the policy gradient. Third, to solve the problem that air combat’s reward function is too sparse, we use MaxEnt IRL to design a shaping reward to accelerate the convergence of SA-DDPG.

Aircraft Upset Recovery Strategy and Pilot Assistance System Based on Reinforcement Learning

Two-Stage Strategy to Achieve a Reinforcement Learning-Based Upset Recovery Policy for Aircraft

Deep reinforcement learning-based upset recovery control for generic transport aircraft

An Aircraft Collision Avoidance Method Based on Deep Reinforcement Learning

Model Predictive Control Based Washout Algorithm Design for Flight Simulator Upset Prevention and Recovery Training

Pilot Error and Error Recovery

Supervisory Control Strategy for AVs under Unexpected Emergency Conditions

Safely Learn to Fly Aircraft from Human: an Offline–Online Reinforcement Learning Strategy and Its Application to Aircraft Stall Recovery

Reinforcement Learning–Based Adaptive Attitude Control Method for a Class of Hypersonic Flight Vehicles Subject to Nonaffine Structure and Unmatched Disturbances

UAV Autonomous Aerial Combat Maneuver Strategy Generation with Observation Error Based on State-Adversarial Deep Deterministic Policy Gradient and Inverse Reinforcement Learning

Fuzzy Recovery Controller for Deep Stall Based on Particle Swarm Optimization

Reinforcement Learning-based Hierarchical Obstacle Avoidance Strategy for Fixed-wing Aircraft

Safety Control Design with Flight Envelope Protection and Reference Command Generation.

A Simulation Based Intelligent Analysis Framework of Aircraft Reliability, Resilience and Vulnerability

Coupling Evaluation Model of Abnormal Flight Recovery Strategy

Dynamic Obstacle Avoidance of Fixed-wing Aircraft in Final Phase Via Reinforcement Learning

Model-Reference Reinforcement Learning for Safe Aerial Recovery of Unmanned Aerial Vehicles

Decision Support System For The Irregular Flight Recovery Problem

Modeling Fuzzy and Adaptive Human Behavior for Aircraft with Dynamic-Pitch-Control Envelope Cue

Fault-Tolerant Control Strategy for Aircraft with Asymmetric Structure Damage

Learn-to-Recover: Retrofitting UAVs with Reinforcement Learning-Assisted Flight Control under Cyber-Physical Attacks