Abstract:Autonomous decision-making in unmanned aerial vehicle (UVA) confrontations presents challenges in making optimal strategy. Therefore, deep reinforcement learning (DRL) has been adopted to address these issues. However, existing DRL decision-making models suffer from poor situational awareness and inability to distinguish between different intentions. Therefore, a multi-intent autonomous decision-making is proposed in this paper. First, three typical intentions are designed comprising head-on attacking, pursuing and fleeing to derive decision models representing different intentions. Reinforcement learning based air combat game model is constructed with different intentions, which contains designing reward functions for intentions to deal with the problem of sparse rewards. Then, we propose the Temporal Proximal Policy Optimization (T-PPO) algorithm, which optimizes the Proximal Policy Optimization algorithm by integrating the long short-term memory network and feedforward neural network. This algorithm extracts the historical temporal information to enhance situational awareness. In addition, a basic-confrontation progressive training method is proposed to provide intention guidance and increase training diversity, which can improve learning efficiency and intelligent decision-making capability. Finally, experiments in our constructed UAV confrontation environment demonstrate that the proposed intentional decision models exhibit good performance in stability and learning efficiency, achieving high rewards, win rates, and low steps. Specifically, our autonomous decision-making increases win rate by 26% when head-on attacking and learning efficiency by 50% when pursuing. It is further proof of the potential and value of our multi-intent autonomous decision-making applications.

Deep Reinforcement Learning-based Behaviour Generation Algorithm for Air Combat Escape Intention

Large Scale Pursuit-Evasion under Collision Avoidance Using Deep Reinforcement Learning.

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Research on Intelligent Evasion Methods for UAV Based on Deep Reinforcement Learning

Pursuit-Evasion Games for Multi-agent Based on Reinforcement Learning with Obstacles

Curriculum-RL Based Air Combat Decision-Making

Air Combat Maneuver Decision Based on Deep Reinforcement Learning and Game Theory

Air-Combat Strategy Using Deep Q-Learning

An intelligent generating method for multi-target attacking strategy based on environment-aware deep reinforcement learning

Improving Maneuver Strategy in Air Combat by Alternate Freeze Games with a Deep Reinforcement Learning Algorithm

Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning

Air Combat Maneuver Decision Based on Deep Reinforcement Learning with Auxiliary Reward

Learning Evasion Strategy in Pursuit-Evasion by Deep Q-network

Deep Reinforcement-Learning-Based Air-Combat-Maneuver Generation Framework

Pursuit-evasion Game Strategy of USV Based on Deep Reinforcement Learning in Complex Multi-Obstacle Environment

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

Generation Method of Autonomous Evasive Maneuver Strategy in Air Combat

A Deep Reinforcement Learning Based Intelligent Decision Method for UCAV Air Combat

Probabilistic Reward-Based Reinforcement Learning for Multi-Agent Pursuit and Evasion

Multi-intent autonomous decision-making for air combat with deep reinforcement learning

A Deep Reinforcement Learning Method for Lion and Man Problem