Abstract:Highly intelligent Unmanned Combat Aerial Vehicle (UCAV) formation is expected to bring out strengths in Beyond-Visual-Range (BVR) air combat. Although Multi-Agent Reinforcement Learning (MARL) shows outstanding performance in cooperative decision-making, it is challenging for existing MARL algorithms to quickly converge to an optimal strategy for UCAV formation in BVR air combat where confrontation is complicated and reward is extremely sparse and delayed. Aiming to solve this problem, this paper proposes an Advantage Highlight Multi-Agent Proximal Policy Optimization (AHMAPPO) algorithm. First, at every step, the AHMAPPO records the degree to which the best formation exceeds the average of formations in parallel environments and carries out additional advantage sampling according to it. Then, the sampling result is introduced into the updating process of the actor network to improve its optimization efficiency. Finally, the simulation results reveal that compared with some state-of-the-art MARL algorithms, the AHMAPPO can obtain a more excellent strategy utilizing fewer sample episodes in the UCAV formation BVR air combat simulation environment built in this paper, which can reflect the critical features of BVR air combat. The AHMAPPO can significantly increase the convergence efficiency of the strategy for UCAV formation in BVR air combat, with a maximum increase of 81.5% relative to other algorithms.

Heuristic Particle Swarm Optimization Algorithm of Multi-UCAV Cooperative Attacking Logic

Multi UCAVs Coordinated Attack Target Assignment Using an Improved PSO Algorithm

Pso-Based Multi Ucavs Cooperative Attack Tasks Allocation And Its Simulation

Target Assignment in Cooperative Attacking of UCAVs Based on Multi-Intelligence Improved Glowworm Swarm Optimization Algorithm

A Hybrid Multi-objective Optimal Approach to Multiple UCAVs Coordinated Planning

Optimal Formation Reconfiguration Control of Multiple UCAVs Using Improved Particle Swarm Optimization

Particle Swarm Optimization For A Multi-Ucav Cooperative Task Scheduling

HEURISTIC PARTICLE SWARM OPTIMIZATION ALGORITHM FOR AIR COMBAT DECISION-MAKING ON CMTA

Force Demand Planning of UAV Cluster Adversarial via Hybrid Group Swarm Optimization

Study on Task Allocation Model and Algorithm for multi-UCAV Cooperative Control

Hybrid genetic algorithm based method for multiple cooperative UCAVs mission assigning

?Hybrid Particle Swarm Optimization and Genetic Algorithm for Multi-UAV Formation Reconfiguration

Multi-UCAVs task assignment simulation platform based on parallel ant colony optimization

UCAV Path Planning Based on Ant Colony Optimization and Satisficing Decision Algorithm.

Multiple Ucavs Target Assignment Via Bloch Quantum-Behaved Pigeon-Inspired Optimization

A New Approach to Solve the Mission Assignment Problem for Cooperative UCAVs Using Immune Particle Swarm Optimizations

Cooperative Path Planning for Multiple UCAVs Using an AIS-ACO Hybrid Approach

Decision-making of Multi-Uav Combat Game Via Enhanced Competitive Learning Pigeon-Inspired Optimization

Cooperative target assignment for unmanned combat aerial vehicles based on Bayesian optimization algorithm with decision graphs

Cooperative Decision-Making Algorithm with Efficient Convergence for UCAV Formation in Beyond-Visual-range Air Combat Based on Multi-Agent Reinforcement Learning

A GA-ACO Hybrid Algorithm for the Multi-Uav Mission Planning Problem.