Abstract:Artificial intelligence plays a pivotal role in autonomous decision-making and control within visual range in air combat. Conventional maneuver decision methods that require an exact dynamic model are limited to the point-mass model aircraft. Model-free reinforcement learning enables air combat research to address practical situations featuring high-fidelity nonlinear flight dynamic models. This study addresses the problem of how an agent can maneuver and control in unseen tasks, which is crucial for improving the adaptability of the agent to complex situations that may not be encountered during training. Accordingly, a meta-reinforcement learning framework is first introduced in air combat decisions for training the agent to control the offensive situation and knock down bandits. In particular, the meta-training tasks are constructed based on typical engagement situations in order to learn basic fighter maneuver in air combat. The corresponding reward function for each of these tasks is shaped based on the potential function of situation assessment. In contrast, the meta-evaluation tasks are designed with uniform but harder goal conditions, which require the integration of meta-training. In implementation, a meta-reinforcement learning algorithm is proposed in order to facilitate fast adaptation to multiple tasks. In the context of meta-evaluation tasks, the success rate of our algorithm is approximately 80%. Furthermore, comparative results demonstrate the effectiveness of our method. An ablated study additionally identifies the efficiency of the meta-reinforcement learning framework. Visualization and analysis of the air combat process are presented. Finally, it can be concluded that the unseen tasks are performed with the combinatorial skills learned in meta-training in the context of air combat issues.

Missile-Target Situation Assessment Model Based on Reinforcement Learning

Medium and Far Range Ground-to-Air Missile System Effectiveness Evaluation Model

Intelligent Game Strategies in Target-Missile-Defender Engagement Using Curriculum-Based Deep Reinforcement Learning

Decision-Making Strategies for Close-Range Air Combat Based on Reinforcement Learning with Variable-Scale Actions

Autonomous UAV maneuvering decisions by refining opponent strategies

Learning and Fast Adaptation for Air Combat Decision with Improved Deep Meta-reinforcement Learning

Predictive air combat decision model with segmented reward allocation

Air Combat Maneuver Decision Based on Deep Reinforcement Learning and Game Theory

Supervised Machine Learning for Effective Missile Launch Based on Beyond Visual Range Air Combat Simulations

Real‐time receding horizon pursuit and evasion games of missile guidance based on neural network

Maneuver Decision-Making For Autonomous Air Combat Through Curriculum Learning And Reinforcement Learning With Sparse Rewards

Simulation of air-to-air missile attack ability in air combat

Multi-intent autonomous decision-making for air combat with deep reinforcement learning

Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning

Dynamic Weapon-Target Assignment for Active Protection of Aircraft

UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

Integrated and Adaptive Guidance and Control for Endoatmospheric Missiles via Reinforcement Learning

General multi-agent reinforcement learning integrating adaptive manoeuvre strategy for real-time multi-aircraft conflict resolution

Hierarchical Reinforcement Learning from Competitive Self-play for Dual-aircraft formation air combat

Maneuver Decision-Making Through Automatic Curriculum Reinforcement Learning Without Handcrafted Reward functions

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System