Abstract:With the recent advances in machine learning, creating agents that behave realistically in simulated air combat has become a growing field of interest. This survey explores the application of machine learning techniques for modeling air combat behavior, motivated by the potential to enhance simulation-based pilot training. Current simulated entities tend to lack realistic behavior, and traditional behavior modeling is labor-intensive and prone to loss of essential domain knowledge between development steps. Advancements in reinforcement learning and imitation learning algorithms have demonstrated that agents may learn complex behavior from data, which could be faster and more scalable than manual methods. Yet, making adaptive agents capable of performing tactical maneuvers and operating weapons and sensors still poses a significant challenge. The survey examines applications, behavior model types, prevalent machine learning methods, and the technical and human challenges in developing adaptive and realistically behaving agents. Another challenge is the transfer of agents from learning environments to military simulation systems and the consequent demand for standardization. Four primary recommendations are presented regarding increased emphasis on beyond-visual-range scenarios, multi-agent machine learning and cooperation, utilization of hierarchical behavior models, and initiatives for standardization and research collaboration. These recommendations aim to address current issues and guide the development of more comprehensive, adaptable, and realistic machine learning-based behavior models for air combat applications.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to explore how to use machine - learning techniques to model air - combat behaviors in order to improve the pilot training effect in simulation training. Specifically, the paper focuses on the following aspects: 1. **Modeling of realistic behaviors**: - Current simulated entities (such as computer - generated forces, CGFs) often lack authenticity in behavior, which makes it difficult for pilots to obtain a realistic experience during training. - Traditional behavior - modeling methods are not only labor - intensive but also prone to losing important domain knowledge during the development process. 2. **Adaptive agents**: - By using advanced machine - learning algorithms such as reinforcement learning (RL) and imitation learning (IL), agents can be trained to learn complex air - combat behaviors from data. - These algorithms can generate behavior models more quickly and efficiently, but still face challenges, such as how to enable agents to perform complex tasks such as tactical maneuvers, operating weapons, and sensors. 3. **Multi - agent learning**: - The air - combat environment usually involves interactions among multiple agents, so multi - agent learning (MARL) is an important research direction. - How to coordinate the behaviors of multiple learning agents in a shared environment and how to find a balance between centralized training and decentralized execution are the current research focuses. 4. **Standardization and cooperation**: - Transferring the learned agents from the learning environment to military simulation systems requires solving the standardization problem. - It is necessary to formulate standards and promote research cooperation to ensure the compatibility and interoperability between different systems. 5. **Practical applications**: - Realistic behavior models are not only beneficial for pilot training but can also be applied in fields such as mission planning and tactical development. - Simulation and modeling tools can help mission planners predict and evaluate the results of different scenarios, thereby optimizing strategies and tactics before actual missions. ### Main suggestions The paper proposes four main suggestions to solve the current problems and guide future research directions: 1. **Increase the focus on beyond - visual - range scenarios**: - Beyond - visual - range scenarios are becoming more and more important in modern air combat and require more research and application. 2. **Multi - agent machine - learning and cooperation**: - Emphasize the importance of multi - agent learning and cooperation to better simulate complex air - combat environments. 3. **Use hierarchical behavior models**: - Hierarchical behavior models can represent complex air - combat behaviors more effectively and avoid the state - explosion problem. 4. **Standardization and research cooperation**: - Promote standardization and research cooperation to enhance the compatibility and interoperability between different systems. Through these research and suggestions, the paper hopes to promote the development of more comprehensive, adaptable, and realistic machine - learning - based air - combat behavior models.

A survey of air combat behavior modeling using machine learning

Learning Environment for the Air Domain (LEAD)

Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering

Knowledge-based and data-driven behavioral modeling techniques in engagement simulation

A Survey of Behavior Learning Applications in Robotics -- State of the Art and Perspectives

Behavior modeling method and its application in combat simulation

Mastering air combat game with deep reinforcement learning

Adaptive Synthetic Characters for Military Training

Learning and Fast Adaptation for Air Combat Decision with Improved Deep Meta-reinforcement Learning

Over-the-Horizon Air Combat Environment Modeling and Deep Reinforcement Learning Application

Scaling Intelligent Agents in Combat Simulations for Wargaming

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

Hierarchical Reinforcement Learning from Competitive Self-play for Dual-aircraft formation air combat

Multi-agent collaboration environment simulation

Modeling agent decision and behavior in the light of data science and artificial intelligence

An evolutionary multi-agent reinforcement learning algorithm for multi-UAV air combat

Deep Reinforcement-Learning-Based Air-Combat-Maneuver Generation Framework

Reinforcement Learning-Based Drone Simulators: Survey, Practice, and Challenge

Maneuver Decision-Making For Autonomous Air Combat Through Curriculum Learning And Reinforcement Learning With Sparse Rewards

A survey on modeling for behaviors of complex intelligent systems based on generative adversarial networks

Autonomous Agent for Beyond Visual Range Air Combat: A Deep Reinforcement Learning Approach