Abstract:Autonomous maneuver decision making is the core of intelligent warfare, which has become the main research direction to enable unmanned aerial vehicles (UAVs) to independently generate control commands and complete air combat tasks according to environmental situation information. In this paper, an autonomous maneuver decision making method is proposed for air combat by two cooperative UAVs, which is showcased by using the typical olive formation strategy as a practical example. First, a UAV situation assessment model based on the relative situation is proposed, which uses the real-time target and UAV location information to assess the current situation or threat. Second, the continuous air combat state space is discretized into a 13 dimensional space for dimension reduction and quantitative description, and 15 typical action commands instead of a continuous control space are designed to reduce the difficulty of UAV training. Third, a reward function is designed based on the situation assessment which includes the real-time gain due to maneuver and the final combat winning/losing gain. Fourth, an improved training data sampling strategy is proposed, which samples the data in the experience pool based on priority to accelerate the training convergence. Fifth, a hybrid autonomous maneuver decision strategy for dual-UAV olive formation air combat is proposed which realizes the UAV capability of obstacle avoidance, formation and confrontation. Finally, the air combat task of dual-UAV olive formation is simulated and the results show that the proposed method can help the UAVs defeat the enemy effectively and outperforms the deep Q network (DQN) method without priority sampling in terms of the convergence speed.

A Method of UAV Formation Transformation Based on Reinforcement Learning Multi-agent

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

A Method of Multi-UAV Cooperative Task Assignment Based on Reinforcement Learning

Multi-UAV Behavior-based Formation with Static and Dynamic Obstacles Avoidance via Reinforcement Learning

Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning

Optimal Formation Control and Visual Simulation for Multiple Unmanned Aerial Vehicles (Uavs)

Flight Test of Autonomous Formation Management for Multiple Fixed-Wing UAVs Based on Missile Parallel Method

UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

Multi-agent Reinforcement Learning for a Special Formation Problem

Research on the Multiagent Joint Proximal Policy Optimization Algorithm Controlling Cooperative Fixed-Wing UAV Obstacle Avoidance

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

An evolutionary multi-agent reinforcement learning algorithm for multi-UAV air combat

Distributed Circle Formation Control for Quadrotors Based on Multi-agent Deep Reinforcement Learning

UAV Swarm Confrontation Using Hierarchical Multiagent Reinforcement Learning

Multi-UAV Collaborative Search and Strike based on Reinforcement Learning

Autonomous Maneuver Decision Making of Dual-UAV Cooperative Air Combat Based on Deep Reinforcement Learning

UAV-enabled Collaborative Beamforming via Multi-Agent Deep Reinforcement Learning

Autonomous and cooperative control of UAV cluster with multi-agent reinforcement learning

Enhancing multi-UAV air combat decision making via hierarchical reinforcement learning

Cooperative Sensing Enhanced UAV Path-Following and Obstacle Avoidance with Variable Formation

Research on Formation Keeping of Multi-rotor UAVs Based on Improved Virtual Structure Method