Abstract:The game of pursuit–evasion has always been a popular research subject in the field of Unmanned Aerial Vehicles (UAVs). Current evasion decision making based on reinforcement learning is generally trained only for specific pursuers, and it has limited performance for evading unknown pursuers and exhibits poor generalizability. To enhance the ability of an evasion policy learned by reinforcement learning (RL) to evade unknown pursuers, this paper proposes a pursuit UAV attitude estimation and pursuit strategy identification method and a Model Reference Policy Adaptation (MRPA) algorithm. Firstly, this paper constructs a Markov decision model for the pursuit–evasion game of UAVs that includes the pursuer’s attitude and trains an evasion policy for a specific pursuit strategy using the Soft Actor–Critic (SAC) algorithm. Secondly, this paper establishes a novel relative motion model of UAVs in pursuit–evasion games under the assumption that proportional guidance is used as the pursuit strategy, based on which the pursuit UAV attitude estimation and pursuit strategy identification algorithm is proposed to provide adequate information for decision making and policy adaptation. Furthermore, a Model Reference Policy Adaptation (MRPA) algorithm is presented to improve the generalizability of the evasion policy trained by RL in certain environments. Finally, various numerical simulations imply the precision of pursuit UAV attitude estimation and the accuracy of pursuit strategy identification. Also, the ablation experiment verifies that the MRPA algorithm can effectively enhance the performance of the evasion policy to deal with unknown pursuers.

A Pursuit-Evasion Game on a Real-City Virtual Simulation Platform Based on Multi-Agent Reinforcement Learning

Large Scale Pursuit-Evasion under Collision Avoidance Using Deep Reinforcement Learning.

Pursuit-Evasion Games for Multi-agent Based on Reinforcement Learning with Obstacles

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Evade Unknown Pursuer Via Pursuit Strategy Identification and Model Reference Policy Adaptation (MRPA) Algorithm

Research on Intelligent Evasion Methods for UAV Based on Deep Reinforcement Learning

Multi-UAV Pursuit-Evasion with Online Planning in Unknown Environments by Deep Reinforcement Learning

Pursuit-evasion Game Strategy of USV Based on Deep Reinforcement Learning in Complex Multi-Obstacle Environment

Game of Drones: Multi-UAV Pursuit-Evasion Game With Online Motion Planning by Deep Reinforcement Learning

Distributed Pursuit-Evasion Game of Limited Perception USV Swarm Based on Multiagent Proximal Policy Optimization

Cooperatively Pursuing a Target Unmanned Aerial Vehicle by Multiple Unmanned Aerial Vehicles Based on Multiagent Reinforcement Learning

Learning to Play Pursuit-Evasion with Dynamic and Sensor Constraints

Pursuit-Evasion Game of Unmanded Surface Vehicles Based on Deep Reinforcement Learning

A Pursuit-evasion Game Between Unmanned Aerial Vehicles

Cooperative Pursuit of Unauthorized UAVs in Urban Airspace Via Multi-agent Reinforcement Learning

Multi-Target Pursuit by a Decentralized Heterogeneous UAV Swarm using Deep Multi-Agent Reinforcement Learning

Adaptive Double Fuzzy Systems Based Q-Learning for Pursuit-Evasion Game

Mean Field Game and Decentralized Intelligent Adaptive Pursuit Evasion Strategy for Massive Multi-Agent System under Uncertain Environment

Safety-Critical Control for Multi-Agent Collaborative Pursuit

Multi-Agent Cooperative-Competitive Environment with Reinforcement Learning

Real-time Virtual Simulation Platform for Multi-UVA Hunting Target Using Deep Reinforcement Learning