Abstract:The game of pursuit–evasion has always been a popular research subject in the field of Unmanned Aerial Vehicles (UAVs). Current evasion decision making based on reinforcement learning is generally trained only for specific pursuers, and it has limited performance for evading unknown pursuers and exhibits poor generalizability. To enhance the ability of an evasion policy learned by reinforcement learning (RL) to evade unknown pursuers, this paper proposes a pursuit UAV attitude estimation and pursuit strategy identification method and a Model Reference Policy Adaptation (MRPA) algorithm. Firstly, this paper constructs a Markov decision model for the pursuit–evasion game of UAVs that includes the pursuer’s attitude and trains an evasion policy for a specific pursuit strategy using the Soft Actor–Critic (SAC) algorithm. Secondly, this paper establishes a novel relative motion model of UAVs in pursuit–evasion games under the assumption that proportional guidance is used as the pursuit strategy, based on which the pursuit UAV attitude estimation and pursuit strategy identification algorithm is proposed to provide adequate information for decision making and policy adaptation. Furthermore, a Model Reference Policy Adaptation (MRPA) algorithm is presented to improve the generalizability of the evasion policy trained by RL in certain environments. Finally, various numerical simulations imply the precision of pursuit UAV attitude estimation and the accuracy of pursuit strategy identification. Also, the ablation experiment verifies that the MRPA algorithm can effectively enhance the performance of the evasion policy to deal with unknown pursuers.

Pursuit and evasion game between UVAs based on multi-agent reinforcement learning

Large Scale Pursuit-Evasion under Collision Avoidance Using Deep Reinforcement Learning.

Pursuit-Evasion Games for Multi-agent Based on Reinforcement Learning with Obstacles

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Integral Reinforcement Learning Based Dynamic Stackelberg Pursuit-Evasion Game for Unmanned Surface Vehicles

Pursuit-evasion Game Strategy of USV Based on Deep Reinforcement Learning in Complex Multi-Obstacle Environment

Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning

Cooperatively Pursuing a Target Unmanned Aerial Vehicle by Multiple Unmanned Aerial Vehicles Based on Multiagent Reinforcement Learning

Game of Marine Robots: USV Pursuit Evasion Game Using Online Reinforcement Learning

Adaptive Double Fuzzy Systems Based Q-Learning for Pursuit-Evasion Game

Evade Unknown Pursuer Via Pursuit Strategy Identification and Model Reference Policy Adaptation (MRPA) Algorithm

A Pursuit-evasion Game Between Unmanned Aerial Vehicles

Pursuit-Evasion Game of Multiple Unmannd Surface Vessels in Partially Observable Environments

Pursuit-Evasion Game of Unmanded Surface Vehicles Based on Deep Reinforcement Learning

A Multi-drones Target Tracing Strategy Based on the Pursuit-Evasion Game Formula

Many-to-One Pursuit-Evasion Differential Game Method for UAVs Using Sequential Convex Programming

Multi-Agent Cooperative-Competitive Environment with Reinforcement Learning

Data-Driven Cooperative Multi-Task Assignment in Heterogeneous Multi-Agent Pursuit-Evade Game

Cooperative Pursuit with Multiple Pursuers based on Deep Minimax Q-learning

Adaptive Optimal Control via Q-Learning for Multi-Agent Pursuit-Evasion Games

Optimal avoidance strategy based on nonlinear approximate analytic solution of non-cooperative differential game