Abstract:Autonomous collision avoidance technology provides an intelligent method for unmanned surface vehicles’ (USVs) safe and efficient navigation. In this paper, the USV collision avoidance problem under the constraint of the international regulations for preventing collisions at sea (COLREGs) was studied. Here, a reinforcement learning collision avoidance (RLCA) algorithm is proposed that complies with USV maneuverability. Notably, the reinforcement learning agent does not require any prior knowledge about USV collision avoidance from humans to learn collision avoidance motions well. The double-DQN method was used to reduce the overestimation of the action-value function. A dueling network architecture was adopted to clearly distinguish the difference between a great state and an excellent action. Aiming at the problem of agent exploration, a method based on the characteristics of USV collision avoidance, the category-based exploration method, can improve the exploration ability of the USV. Because a large number of turning behaviors in the early steps may affect the training, a method to discard some of the transitions was designed, which can improve the effectiveness of the algorithm. A finite Markov decision process (MDP) that conforms to the USVs’ maneuverability and COLREGs was used for the agent training. The RLCA algorithm was tested in a marine simulation environment in many different USV encounters, which showed a higher average reward. The RLCA algorithm bridged the divide between USV navigation status information and collision avoidance behavior, resulting in successfully planning a safe and economical path to the terminal.

Fuzzy Categorical Deep Reinforcement Learning of a Defensive Game for an Unmanned Surface Vessel

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

Research on Unmanned Surface Vehicle Collision Avoidance Based on Deep Reinforcement Learning

Unmanned Surface Vessel Motion Intention Recognition Based on Deep Reinforcement Learning

Real-time Planning and Collision Avoidance Control Method Based on Deep Reinforcement Learning

AUV Collision Avoidance Strategy Based on Fuzzy Reinforcement Learning

Speed and Heading Control of an Unmanned Surface Vehicle Using Deep Reinforcement Learning

Deep reinforcement learning-based controller for dynamic positioning of an unmanned surface vehicle

Pursuit-Evasion Game of Unmanded Surface Vehicles Based on Deep Reinforcement Learning

Path Following Control for Unmanned Surface Vehicles: A Reinforcement Learning-Based Method with Experimental Validation.

Model-based Deep Reinforcement Learning for Data-Driven Motion Control of an Under-Actuated Unmanned Surface Vehicle: Path Following and Trajectory Tracking

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances.

Intelligent Controller for Unmanned Surface Vehicles by Deep Reinforcement Learning

Concise Deep Reinforcement Learning Obstacle Avoidance for Underactuated Unmanned Marine Vessels.

Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach

Efficient Steering Control for Long-Term Unmanned Surface Vehicle

A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV

Deep Reinforcement Learning-Based Path Planning of Underactuated Surface Vessels

A Novel Reinforcement Learning Collision Avoidance Algorithm for USVs Based on Maneuvering Characteristics and COLREGs

A Learning Method for AUV Collision Avoidance Through Deep Reinforcement Learning