Abstract:The advancement of hardware and software technology makes multiple cooperative unmanned surface vehicles (USVs) utilized in maritime escorting with low cost and high efficiency. USVs can work as edge computing devices to locally and cooperatively perform heavy computational tasks without dependence of remote cloud servers. As such, we organize a team of USVs to escort a high value ship (e.g., mother ship) in a complex maritime environment with hostile intruder ships, where the significant challenge is to learn cooperation of USVs and assign each USV tasks to achieve optimal performance. To this end, in this paper, a hierarchical scheme is proposed for the USV team to guard a valuable ship, which belongs to problems of sparse rewards and long-time horizons in multi-agent reinforcement learning. The core idea utilized in the proposed scheme is centralized training with decentralized execution, where USVs learn policies to guard a high-value ship with extra shared environmental data from other USVs through communication. Specifically, the ships are divided into two groups, i.e., high-level ship and low-level USVs. The high-level ship optimizes decision-level policy to predict intercept points, while each low-level USV utilizes multi-agent reinforcement learning to learn task-level policy to intercept intruders. The hierarchical task guidance is exploited in maritime escorting, whereby high-level ship's decision-level policy guides the training and execution of task-level policies of USVs. Simulation results and experiment analysis show that the proposed hierarchical scheme can efficiently execute the escort mission.

The Object-Oriented Dynamic Task Assignment for Unmanned Surface Vessels.

Dynamic Task Assignment of Multiple Heterogeneous Unmanned Aerial Vehicles Based on Consensus with Uncertainties

Dynamic Target Assignment Method for UCAV Fleet Using Memory Enhanced Dynamic UMDA

Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning

Multi UCAVs Coordinated Attack Target Assignment Using an Improved PSO Algorithm

An Effective Strategy for Distributed Unmanned Underwater Vehicles to Encircle and Capture Intelligent Targets

UAV-USV Cooperative Task Allocation for Smart Ocean Networks.

A Task Assignment Strategy for Multi-AUV Collaborative Hunting Problem

Constrained Multi-Objective Task Assignment for Uuvs Using Multiple Ant Colonies System

Data Driven Hybrid Edge Computing-Based Hierarchical Task Guidance for Efficient Maritime Escorting with Multiple Unmanned Surface Vehicles

Correction Redistribution Mechanism Based on Forward-Reverse Solutions and Real-Time Path Dynamic Adaptive Re-Planning for Multi-AUVs Collaborative Search

A Dynamic Task Allocation Algorithm for Heterogeneous UUV Swarms

A multi-objective bi-level task planning strategy for UUV target visitation in ocean environment

Task Allocation With Unmanned Surface Vehicles in Smart Ocean IoT

An adaptive distributed auction algorithm and its application to multi-AUV task assignment

Control Strategies for Target-Attacker-Defender Games of USVs

Multi-missile Interception for Multi-Targets: Dynamic Situation Assessment, Target Allocation and Cooperative Interception in Groups

Intelligent Task Allocation and Planning for Unmanned Surface Vehicle (USV) Using Self-Attention Mechanism and Locking Sweeping Method

Allocation Strategy Optimization Using Repulsion-Enhanced Quantum Particle Swarm Optimization for Multi-AUV Systems

AUV Antagonistic Tactics Study for Naval Battle Based on Dynamic Game.

Dynamic Task Assignment and Path Planning for Multi-AUV System in Variable Ocean Current Environment