Abstract:Artificial intelligence (AI) has enormous potential to improve Air Force pilot training by providing actionable feedback to pilot trainees on the quality of their maneuvers and enabling instructor-less flying familiarization for early-stage trainees in low-cost simulators. Historically, AI challenges consisting of data, problem descriptions, and example code have been critical to fueling AI breakthroughs. The Department of the Air Force-Massachusetts Institute of Technology AI Accelerator (DAF-MIT AI Accelerator) developed such an AI challenge using real-world Air Force flight simulator data. The Maneuver ID challenge assembled thousands of virtual reality simulator flight recordings collected by actual Air Force student pilots at Pilot Training Next (PTN). This dataset has been publicly released at <a class="link-external link-http" href="http://Maneuver-ID.mit.edu" rel="external noopener nofollow">this http URL</a> and represents the first of its kind public release of USAF flight training data. Using this dataset, we have applied a variety of AI methods to separate "good" vs "bad" simulator data and categorize and characterize maneuvers. These data, algorithms, and software are being released as baselines of model performance for others to build upon to enable the AI ecosystem for flight simulator training.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper attempts to address the issue of identifying flight maneuvers in flight simulator data. Specifically, the goals of the paper include: 1. **Classifying Physically Feasible and Infeasible Data**: - Dividing the data recorded by the flight simulator into "good" (physically feasible) and "bad" (physically infeasible) categories. This includes identifying continuous trajectories, recognizable maneuvers, and actions that do not conform to physical laws. 2. **Identifying Maneuvers Attempted by Pilots**: - Recognizing specific maneuvers attempted by pilots from the flight simulator records. Currently, only examples of single maneuvers are available, which poses challenges for traditional recognition methods. 3. **Evaluating the Quality of Pilot Maneuvers**: - Once the pilot's maneuvers are identified, scoring them to improve the efficiency and quality of pilot training. ### Background and Motivation Flight simulators play a crucial role in pilot training. The current training model requires experienced instructors to teach basic flight maneuvers, leading to a scarcity of resources. By introducing artificial intelligence (AI), real-time feedback can be provided, helping trainees learn flight maneuvers better at an early stage. Therefore, an important step is to teach AI to recognize different categories of flight maneuvers from flight simulator data. ### Methods and Contributions 1. **Data Source**: - The dataset includes records of student training flights in a virtual reality (VR) environment. This data comes from the U.S. Air Force's Pilot Training Next (PTN) program, using Lockheed Martin Prepar3d flight simulator software. 2. **Data Exploration and Preprocessing**: - Exploring the data using visualization tools (such as Blender and Plotly) to identify abnormal behaviors in the data, such as impossible speeds, teleportation, prolonged gliding, etc. - Classifying the data using basic statistical measures, random forest models, convolutional neural networks, and other methods. 3. **Task 1: Classifying "Good" and "Bad" Data**: - Classifying the data using basic statistical measures, classical models, and image classification methods. Among them, random forest and bagged tree models performed best, with an accuracy of about 98%. 4. **Task 2: Identifying Maneuvers**: - Due to the lack of labeled data, the research team adopted time series random forest and unsupervised learning methods. Preliminary results indicate that these methods can still generate useful predictions with limited data. ### Conclusion This paper successfully addresses the issue of identifying flight maneuvers in flight simulator data through various methods and techniques. These achievements not only help improve the efficiency of pilot training but also provide a foundation for future research.

AI Enabled Maneuver Identification via the Maneuver Identification Challenge

Augmenting Flight Training with AI to Efficiently Train Pilots

AAM-Gym: Artificial Intelligence Testbed for Advanced Air Mobility

Manual-Based Automated Maneuvering Decisions for Air-to-Air Combat

Responsible machine learning for united states air force pilot candidate selection

The Development of Rule-based AI Engagement Model for Air-to-Air Combat Simulation

Artificial Intelligence for Real-Time Tolerance to Critical Flight Data Errors in Large Aircraft

Shared Autonomy with IDA: Interventional Diffusion Assistance

Discovering Expert-Level Air Combat Knowledge via Deep Excitatory-Inhibitory Factorized Reinforcement Learning

The Need for Artificial Intelligence Curriculum in Military Medical Education

Driver Maneuver Identification with Multi-Representation Learning and Meta Model Update Designs

Maneuvering Strategy and Own Aircraft Movement Prediction in Trial Input Method - Low Angle of Attack.

An Integrated Imitation and Reinforcement Learning Methodology for Robust Agile Aircraft Control with Limited Pilot Demonstration Data

Multi-agent collaboration environment simulation

From Demonstration to Flight: Realization of Autonomous Aerobatic Maneuvers for Fast, Miniature Fixed-Wing UAVs

Deep Reinforcement-Learning-Based Air-Combat-Maneuver Generation Framework

ASID: Active Exploration for System Identification in Robotic Manipulation

A survey of air combat behavior modeling using machine learning

Vision-Based 3D Aerial Target Detection and Tracking for Maneuver Decision in Close-Range Air Combat

Challenges in Close-Proximity Safe and Seamless Operation of Manned and Unmanned Aircraft in Shared Airspace

UNLEASHING AUTONOMOUS FORCES: INTEGRATING AI-DRIVEN DRONES IN MODERN MILITARY STRATEGY