Abstract:The game of pursuit–evasion has always been a popular research subject in the field of Unmanned Aerial Vehicles (UAVs). Current evasion decision making based on reinforcement learning is generally trained only for specific pursuers, and it has limited performance for evading unknown pursuers and exhibits poor generalizability. To enhance the ability of an evasion policy learned by reinforcement learning (RL) to evade unknown pursuers, this paper proposes a pursuit UAV attitude estimation and pursuit strategy identification method and a Model Reference Policy Adaptation (MRPA) algorithm. Firstly, this paper constructs a Markov decision model for the pursuit–evasion game of UAVs that includes the pursuer’s attitude and trains an evasion policy for a specific pursuit strategy using the Soft Actor–Critic (SAC) algorithm. Secondly, this paper establishes a novel relative motion model of UAVs in pursuit–evasion games under the assumption that proportional guidance is used as the pursuit strategy, based on which the pursuit UAV attitude estimation and pursuit strategy identification algorithm is proposed to provide adequate information for decision making and policy adaptation. Furthermore, a Model Reference Policy Adaptation (MRPA) algorithm is presented to improve the generalizability of the evasion policy trained by RL in certain environments. Finally, various numerical simulations imply the precision of pursuit UAV attitude estimation and the accuracy of pursuit strategy identification. Also, the ablation experiment verifies that the MRPA algorithm can effectively enhance the performance of the evasion policy to deal with unknown pursuers.

High-Speed Three-Dimensional Aerial Vehicle Evasion Based on a Multi-Stage Dueling Deep Q-Network

Large Scale Pursuit-Evasion under Collision Avoidance Using Deep Reinforcement Learning.

Research on Intelligent Evasion Methods for UAV Based on Deep Reinforcement Learning

Intelligent Pursuit–Evasion Game Based on Deep Reinforcement Learning for Hypersonic Vehicles

Adaptive Double Fuzzy Systems Based Q-Learning for Pursuit-Evasion Game

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Intelligent maneuver strategy for hypersonic vehicles in three-player pursuit-evasion games via deep reinforcement learning

Pursuit-Evasion Games for Multi-agent Based on Reinforcement Learning with Obstacles

Deep Reinforcement Learning Based Strategy for Quadrotor UAV Pursuer and Evader Problem

Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning

Evade Unknown Pursuer Via Pursuit Strategy Identification and Model Reference Policy Adaptation (MRPA) Algorithm

Generation Method of Autonomous Evasive Maneuver Strategy in Air Combat

Generalization Strategy Design of UAVs Pursuit Evasion Game Based on DDPG

Pursuit-evasion Game Strategy of USV Based on Deep Reinforcement Learning in Complex Multi-Obstacle Environment

Intelligent Maneuver Strategy for a Hypersonic Pursuit-Evasion Game Based on Deep Reinforcement Learning

Cooperative Pursuit with Multiple Pursuers based on Deep Minimax Q-learning

Learning Evasion Strategy in Pursuit-Evasion by Deep Q-network

A Pursuit-Evasion Game on a Real-City Virtual Simulation Platform Based on Multi-Agent Reinforcement Learning

Game of Drones: Multi-UAV Pursuit-Evasion Game With Online Motion Planning by Deep Reinforcement Learning

A Pursuit-evasion Game Between Unmanned Aerial Vehicles

Maneuvering Decision Making Based on Cloud Modeling Algorithm for UAV Evasion–Pursuit Game