Abstract:Autonomous decision-making in unmanned aerial vehicle (UVA) confrontations presents challenges in making optimal strategy. Therefore, deep reinforcement learning (DRL) has been adopted to address these issues. However, existing DRL decision-making models suffer from poor situational awareness and inability to distinguish between different intentions. Therefore, a multi-intent autonomous decision-making is proposed in this paper. First, three typical intentions are designed comprising head-on attacking, pursuing and fleeing to derive decision models representing different intentions. Reinforcement learning based air combat game model is constructed with different intentions, which contains designing reward functions for intentions to deal with the problem of sparse rewards. Then, we propose the Temporal Proximal Policy Optimization (T-PPO) algorithm, which optimizes the Proximal Policy Optimization algorithm by integrating the long short-term memory network and feedforward neural network. This algorithm extracts the historical temporal information to enhance situational awareness. In addition, a basic-confrontation progressive training method is proposed to provide intention guidance and increase training diversity, which can improve learning efficiency and intelligent decision-making capability. Finally, experiments in our constructed UAV confrontation environment demonstrate that the proposed intentional decision models exhibit good performance in stability and learning efficiency, achieving high rewards, win rates, and low steps. Specifically, our autonomous decision-making increases win rate by 26% when head-on attacking and learning efficiency by 50% when pursuing. It is further proof of the potential and value of our multi-intent autonomous decision-making applications.

Air-to-ground Shepherd Problem: an Action-Delay Reinforcement Learning Approach

Hierarchical Decision and Control for Continuous Multitarget Problem: Policy Evaluation with Action Delay

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Learning to Herd Agents Amongst Obstacles: Training Robust Shepherding Behaviors Using Deep Reinforcement Learning

Relevant experience learning: A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

Path Planning of Unmanned Aerial Vehicle in Complex Environments Based on State-Detection Twin Delayed Deep Deterministic Policy Gradient

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

Group-Based Deep Reinforcement Learning in Multi-UAV Confrontation

Effective Robotic Swarm Shepherding in the Presence of Obstacles

Joint Optimization of Multi-UAV Deployment and User Association Via Deep Reinforcement Learning for Long-Term Communication Coverage

Confrontation and Obstacle-Avoidance of Unmanned Vehicles Based on Progressive Reinforcement Learning.

Trajectory Design and Access Control for Air–Ground Coordinated Communications System With Multiagent Deep Reinforcement Learning

Multi-intent autonomous decision-making for air combat with deep reinforcement learning

Time-Effective UAV-IRS-Collaborative Data Harvesting: A Robust Deep Reinforcement Learning Approach

Multi-UAV Assisted Offloading Optimization: A Game Combined Reinforcement Learning Approach

Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning

Reactive shepherding along a dynamic path

Asynchronous Curriculum Experience Replay: A Deep Reinforcement Learning Approach for UAV Autonomous Motion Control in Unknown Dynamic Environments

Scheduling Drone and Mobile Charger via Hybrid-Action Deep Reinforcement Learning

Continuous Deep Hierarchical Reinforcement Learning for Ground-Air Swarm Shepherding

Over-the-Horizon Air Combat Environment Modeling and Deep Reinforcement Learning Application