Abstract:Autonomous maneuver decision making is the core of intelligent warfare, which has become the main research direction to enable unmanned aerial vehicles (UAVs) to independently generate control commands and complete air combat tasks according to environmental situation information. In this paper, an autonomous maneuver decision making method is proposed for air combat by two cooperative UAVs, which is showcased by using the typical olive formation strategy as a practical example. First, a UAV situation assessment model based on the relative situation is proposed, which uses the real-time target and UAV location information to assess the current situation or threat. Second, the continuous air combat state space is discretized into a 13 dimensional space for dimension reduction and quantitative description, and 15 typical action commands instead of a continuous control space are designed to reduce the difficulty of UAV training. Third, a reward function is designed based on the situation assessment which includes the real-time gain due to maneuver and the final combat winning/losing gain. Fourth, an improved training data sampling strategy is proposed, which samples the data in the experience pool based on priority to accelerate the training convergence. Fifth, a hybrid autonomous maneuver decision strategy for dual-UAV olive formation air combat is proposed which realizes the UAV capability of obstacle avoidance, formation and confrontation. Finally, the air combat task of dual-UAV olive formation is simulated and the results show that the proposed method can help the UAVs defeat the enemy effectively and outperforms the deep Q network (DQN) method without priority sampling in terms of the convergence speed.

A UAV Maneuver Decision-Making Algorithm for Autonomous Airdrop Based on Deep Reinforcement Learning

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

UAV maneuver decision-making via deep reinforcement learning for short-range air combat

Application of Deep Reinforcement Learning in UAVs: A Review

Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

Autonomous Decision-Making Method for Combat Mission of UAV based on Deep Reinforcement Learning

Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning

Autonomous Maneuver Decision Making of Dual-UAV Cooperative Air Combat Based on Deep Reinforcement Learning

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

Research on UCAV Maneuvering Decision Method Based on Heuristic Reinforcement Learning

Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm

Deep Reinforcement Learning for UAV Intelligent Mission Planning

Air Combat Maneuver Decision Method Based on A3C Deep Reinforcement Learning

UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

MW-MADDPG: a meta-learning based decision-making method for collaborative UAV swarm

Asynchronous Curriculum Experience Replay: A Deep Reinforcement Learning Approach for UAV Autonomous Motion Control in Unknown Dynamic Environments

Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning

Enhancing multi-UAV air combat decision making via hierarchical reinforcement learning

Deep reinforcement learning-based air combat maneuver decision-making: literature review, implementation tutorial and future direction

Autonomous obstacle avoidance of UAV based on deep reinforcement learning