Abstract:ABSTRACT The effective implementation of multi-unmanned aerial vehicle (UAV) decision making and improvement in the efficiency of coordinated mission execution are currently the top priorities of air combat research. To solve the problem of multi-UAV cooperative air combat maneuvering confrontation,a multi-UAV cooperative air combat maneuvering confrontation decision-making method based on long short-term memory (LSTM) and convolutional deep reinforcement learning of competitive graphs is proposed. First,the problem of multi-UAV cooperative air combat maneuvering confrontation is described. Second,in the deep dueling Q network,the LSTM network is introduced to process air combat information with a strong temporal correlation. Further,a graph convolutional network is built as a communication basis between multiple UAVs and a cooperative air combat training framework based on LSTM,and a convolutional deep reinforcement learning algorithm for the dueling graph is proposed to improve the convergence. In the proposed method,the communication problem between UAVs is transformed into a graph model,where each UAV is regarded as a node,and the observation state of the UAV is regarded as the attribute of a node. The convolutional layer captures the cooperative relationship between each node,and communication between UAVs is realized through information sharing. Subsequently,the extracted air combat feature information with time sequence is inputted into the LSTM and deep dueling Q networks for evaluating action values. The LSTM network can process sequence information and encode historical states into the hidden state of the network so that the network can better capture temporal dependencies and thus predict the value function of the current state better. The simulation resultsshow that when the opponent adopts a nonmaneuvering strategy,the UAV formation developed using the proposed method as the core decision-making strategy can learn a reasonable maneuvering strategy and cooperate to a certain extent when facing an opponent using a fixed strategy. This proves the effectiveness of the algorithm in multi-UAV collaborative air combat maneuvering confrontation problems,enabling UAV formations to achieve teamwork and improve air combat efficiency. In a two-on-one air combat situation,the greedy algorithm is used as the decision-making strategy of the enemy aircraft. The results of simulation comparison experiments show that when faced with opponents using certain rules and strategies,the red team formation can learn reasonable maneuver confrontation strategies and cooperate in the decision-making process to form certain air combat tactics,which improve the combat efficiency of the red team. Compared with the basic method,the proposed method exhibits a more stable learning process and faster decision-making speed for UAV cooperative air combat.

Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles

Long and Short Term Maneuver Trajectory Prediction of UCAV Based on Deep Learning

Autonomous Maneuver Decision of UCAV Air Combat Based on Double Deep Q Network Algorithm and Stochastic Game Theory

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning

A Decision-Making Method for Air Combat Maneuver Based on Hybrid Deep Learning Network

Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm

A Deep Reinforcement Learning Based Intelligent Decision Method for UCAV Air Combat

UAV Countermeasure Maneuver Decision Based on Deep Reinforcement Learning

Air Combat Maneuver Decision Method Based on A3C Deep Reinforcement Learning

Autonomous Maneuver Decision Making of Dual-UAV Cooperative Air Combat Based on Deep Reinforcement Learning

UAV maneuver decision-making via deep reinforcement learning for short-range air combat

The Optimal Strategies of Maneuver Decision in Air Combat of UCAV Based on the Improved TD3 Algorithm

Autonomous Decision-Making Method for Combat Mission of UAV based on Deep Reinforcement Learning

Manoeuvre Decision‐making of Unmanned Aerial Vehicles in Air Combat Based on an Expert Actor‐based Soft Actor Critic Algorithm

UAV Air Game Maneuver Decision-Making Using Dueling Double Deep Q Network with Expert Experience Storage Mechanism

Research on Intelligent Maneuvering Decision in Close Air Combat Based on Deep Q Network

Collaborative Air Combat Maneuvering Decision-Making Method Based on Graph Convolutional Deep Reinforcement Learning

Collaborative Decision-making in Heterogeneous UAV Swarms Based on Multi-agent Deep Reinforcement Learning

Research on UCAV Maneuvering Decision Method Based on Heuristic Reinforcement Learning

Deep reinforcement learning and its application in autonomous fitting optimization for attack areas of UCAVs