Abstract:Autonomous maneuver decision making is the core of intelligent warfare, which has become the main research direction to enable unmanned aerial vehicles (UAVs) to independently generate control commands and complete air combat tasks according to environmental situation information. In this paper, an autonomous maneuver decision making method is proposed for air combat by two cooperative UAVs, which is showcased by using the typical olive formation strategy as a practical example. First, a UAV situation assessment model based on the relative situation is proposed, which uses the real-time target and UAV location information to assess the current situation or threat. Second, the continuous air combat state space is discretized into a 13 dimensional space for dimension reduction and quantitative description, and 15 typical action commands instead of a continuous control space are designed to reduce the difficulty of UAV training. Third, a reward function is designed based on the situation assessment which includes the real-time gain due to maneuver and the final combat winning/losing gain. Fourth, an improved training data sampling strategy is proposed, which samples the data in the experience pool based on priority to accelerate the training convergence. Fifth, a hybrid autonomous maneuver decision strategy for dual-UAV olive formation air combat is proposed which realizes the UAV capability of obstacle avoidance, formation and confrontation. Finally, the air combat task of dual-UAV olive formation is simulated and the results show that the proposed method can help the UAVs defeat the enemy effectively and outperforms the deep Q network (DQN) method without priority sampling in terms of the convergence speed.

Generation Method of Autonomous Evasive Maneuver Strategy in Air Combat

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Research on Intelligent Evasion Methods for UAV Based on Deep Reinforcement Learning

Autonomous Maneuver Decision of UCAV Air Combat Based on Double Deep Q Network Algorithm and Stochastic Game Theory

Evasive Maneuver Strategy for UCAV in Beyond-Visual-Range Air Combat Based on Hierarchical Multi-Objective Evolutionary Algorithm

The Optimal Strategies of Maneuver Decision in Air Combat of UCAV Based on the Improved TD3 Algorithm

Maneuver Strategy Generation of UCAV for Within Visual Range Air Combat Based on Multi-Agent Reinforcement Learning and Target Position Prediction

Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm

Air Combat Maneuver Decision Method Based on A3C Deep Reinforcement Learning

High-Speed Three-Dimensional Aerial Vehicle Evasion Based on a Multi-Stage Dueling Deep Q-Network

Dual-UAVs Maneuvering Strategy Generation Algorithm Based on Cooperative Reward Mechanism and MATD3

One-to-one Air-combat Maneuver Strategy Based on Improved TD3 Algorithm

Deep Reinforcement Learning Based Strategy for Quadrotor UAV Pursuer and Evader Problem

Maneuver and Attack Strategy Generation Method for Autonomous Air Combat in Hybrid Action Space Based on Proximal Policy Optimization

Autonomous Maneuver Decision for Unmanned Fighters in Multi-Agent BVR Air Combat

Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning

Air Combat Maneuver Decision Based on Deep Reinforcement Learning and Game Theory

Autonomous Maneuver Decision Making of Dual-UAV Cooperative Air Combat Based on Deep Reinforcement Learning

One-to-one Close Air Combat Maneuver Decision Method Based on Target Maneuver Intention Prediction

Research on UCAV Maneuvering Decision Method Based on Heuristic Reinforcement Learning

Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles