Abstract:Reinforcement learning has been applied to air combat problems in recent years, and the idea of curriculum learning is often used for reinforcement learning, but traditional curriculum learning suffers from the problem of plasticity loss in neural networks. Plasticity loss is the difficulty of learning new knowledge after the network has converged. To this end, we propose a motivational curriculum learning distributed proximal policy optimization (MCLDPPO) algorithm, through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods. The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide. Furthermore, a complete tactical maneuver is encapsulated based on the existing air combat knowledge, and through the flexible use of these maneuvers, some tactics beyond human knowledge can be realized. In addition, we designed an interruption mechanism for the agent to increase the frequency of decision-making when the agent faces an emergency. When the number of threats received by the agent changes, the current action is interrupted in order to reacquire observations and make decisions again. Using the interruption mechanism can significantly improve the performance of the agent. To simulate actual air combat better, we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously, effectively improving data throughput. The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat, verifying the effectiveness of the algorithmic framework proposed in this paper.

Master-Slave Curriculum Design for Reinforcement Learning

Learning Curriculum Policies for Reinforcement Learning

Towards Skilled Population Curriculum for Multi-Agent Reinforcement Learning

Effective Master-Slave Communication On A Multi-Agent Deep Reinforcement Learning System

Accelerate Multi-Agent Reinforcement Learning in Zero-Sum Games with Subgame Curriculum Learning

Improved Reinforcement Learning with Curriculum

Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning

Curriculum-RL Based Air Combat Decision-Making

Syllabus: Portable Curricula for Reinforcement Learning Agents

An Alternative Curriculum Learning Approach with Macro Actions for Deep Reinforcement Learning

A Curriculum Learning Based Multi-agent Reinforcement Learning Method for Realtime Strategy Game

Proximal Curriculum with Task Correlations for Deep Reinforcement Learning

Proximal Curriculum for Reinforcement Learning Agents

Curriculum Learning with a Progression Function

Mastering air combat game with deep reinforcement learning

CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Models

Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems

Teacher-student curriculum learning for reinforcement learning

Curriculum Learning for Cooperation in Multi-Agent Reinforcement Learning

An Optimization Framework for Task Sequencing in Curriculum Learning

Understanding the Complexity Gains of Single-Task RL with a Curriculum