Abstract:Reinforcement learning has been applied to air combat problems in recent years, and the idea of curriculum learning is often used for reinforcement learning, but traditional curriculum learning suffers from the problem of plasticity loss in neural networks. Plasticity loss is the difficulty of learning new knowledge after the network has converged. To this end, we propose a motivational curriculum learning distributed proximal policy optimization (MCLDPPO) algorithm, through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods. The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide. Furthermore, a complete tactical maneuver is encapsulated based on the existing air combat knowledge, and through the flexible use of these maneuvers, some tactics beyond human knowledge can be realized. In addition, we designed an interruption mechanism for the agent to increase the frequency of decision-making when the agent faces an emergency. When the number of threats received by the agent changes, the current action is interrupted in order to reacquire observations and make decisions again. Using the interruption mechanism can significantly improve the performance of the agent. To simulate actual air combat better, we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously, effectively improving data throughput. The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat, verifying the effectiveness of the algorithmic framework proposed in this paper.

Realizing Midcourse Penetration With Deep Reinforcement Learning

Maneuvering penetration strategies of ballistic missiles based on deep reinforcement learning

Spacecraft Attitude Maneuver Planning Based on Deep Reinforcement Learning under Complex Constraints

Learning-Based Multi-missile Maneuver Penetration Approach

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

High-dynamic Intelligent Maneuvering Guidance Strategy Via Deep Reinforcement Learning

Online Trajectory Planning Method for Midcourse Guidance Phase Based on Deep Reinforcement Learning

Integrated Guidance-and-Control Design for Three-Dimensional Interception Based on Deep-Reinforcement Learning

Battlefield Situation Deduction and Maneuver Decision Using Deep Q-Learning

Robust Motion Control for UAV in Dynamic Uncertain Environments Using Deep Reinforcement Learning.

Intelligent Game Strategies in Target-Missile-Defender Engagement Using Curriculum-Based Deep Reinforcement Learning

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex Unknown Environments

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

Receding Horizon Control Based Real-time Strategy for Missile Pursuit-evasion Game

A Multi-Constraint Guidance and Maneuvering Penetration Strategy via Meta Deep Reinforcement Learning

Intercept Guidance of Maneuvering Targets with Deep Reinforcement Learning

Integrated and Adaptive Guidance and Control for Endoatmospheric Missiles via Reinforcement Learning

Leader-follower Formation Control for a Multi-missile System Via Deep Reinforcement Learning

Deep Learning Midcourse Guidance for Interceptor Missile

Recorded Recurrent Deep Reinforcement Learning Guidance Laws for Intercepting Endoatmospheric Maneuvering Missiles

Mastering air combat game with deep reinforcement learning