Abstract:With the extensive deployment of unmanned systems in various fields, the requirements for unmanned underwater vehicles (UUV) are becoming higher and higher. A single UUV can no longer meet the task requirements, and the cooperative control of multiple UUVs is required. In this paper, facing the hunting-escape task, establishing the kinematics and observation communication model of UUV, the multi-UUV pursuit task is completed based on this model. The multi-UUV maneuvering strategy set, payoff function and Nash equilibrium solution are given by using dynamic extended form game. In the game task, the pursuit UUVs decompose many-to-one game to one-to-one, and the escape UUV calculates the highest income among the multiple UUVs, that is, in the process of the task, the pursuit party simplifies the task to one-to-one interception, and the other is the best strategy for many-to-one selection. Finally, the pursuit and escape simulation experiments are carried out to verify the effectiveness and superiority of the algorithm. From the perspective of effectiveness, after 29 games, the pursuit UUV successfully encircles the escape UUV. The effectiveness of the algorithm is verified from the encircling path, the change of the distance between the pursuit UUV and the breaching party's UUV, and the final situation. The effectiveness of UUV controller is analyzed from lateral, longitudinal velocity and heading control. In terms of the superiority of the algorithm, when the pursuit party chooses the maneuver strategy given by the interception and breakthrough theory, the pursuit party fails after the same time and strategy selection steps, and the escape succeeds. The superiority of the algorithm can be obtained from the comparison of the simulation results.

Multi-AUV Pursuit-Evasion Game in the Internet of Underwater Things: an Efficient Training Framework Via Offline Reinforcement Learning

Learning an End-To-End Policy for AUV Control Within Just Forty Minutes Using Parallel Simulation

Large Scale Pursuit-Evasion under Collision Avoidance Using Deep Reinforcement Learning.

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Pursuit-Evasion Games for Multi-agent Based on Reinforcement Learning with Obstacles

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Pursuit-Evasion Game of Unmanded Surface Vehicles Based on Deep Reinforcement Learning

Pursuit-Evasion Game of Multiple Unmannd Surface Vessels in Partially Observable Environments

Integral Reinforcement Learning Based Dynamic Stackelberg Pursuit-Evasion Game for Unmanned Surface Vehicles

Multi-Objective-Optimization Multi-AUV Assisted Data Collection Framework for IoUT Based on Offline Reinforcement Learning

Pursuit-evasion Game Strategy of USV Based on Deep Reinforcement Learning in Complex Multi-Obstacle Environment

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Differential Game-Based Deep Reinforcement Learning in Underwater Target Hunting Task

Game of Marine Robots: USV Pursuit Evasion Game Using Online Reinforcement Learning

Distributed Pursuit-Evasion Game of Limited Perception USV Swarm Based on Multiagent Proximal Policy Optimization

APPLICATION OF EXTENDED GAME IN MULTI-UUV PURSUIT-ESCAPE TASK

A Pursuit-Evasion Game on a Real-City Virtual Simulation Platform Based on Multi-Agent Reinforcement Learning

Distributional Soft Actor-Critic-Based Multi-AUV Cooperative Pursuit for Maritime Security Protection

Enhancing Underwater IoT Security: A Collaborative Pursuit Strategy Using Multi-Agent Reinforcement Learning

Multi-UAV Pursuit-Evasion with Online Planning in Unknown Environments by Deep Reinforcement Learning

HA-MARL: Heuristic and APF Assisted Multi-Agent Reinforcement Learning for Wireless Data Sharing in AUV Swarms