Abstract:This paper presents a framework to alleviate the Deep Reinforcement Learning (DRL) training data sparsity problem that is present in challenging domains by creating a DRL agent training and vehicle integration methodology. The methodology leverages accessible domains to train an agent to solve navigational problems such as obstacle avoidance and allows the agent to generalize to challenging and inaccessible domains such as those present in marine environments with minimal further training. This is done by integrating a DRL agent at a high level of vehicle control and leveraging existing path planning and proven low-level control methodologies that are utilized in multiple domains. An autonomy package with a tertiary multilevel controller is developed to enable the DRL agent to interface at the prescribed high control level and thus be separated from vehicle dynamics and environmental constraints. An example Deep Q Network (DQN) employing this methodology for obstacle avoidance is trained in a simulated ground environment, and then its ability to generalize across domains is experimentally validated. Experimental validation utilized a simulated water surface environment and real-world deployment of ground and water robotic platforms. This methodology, when used, shows that it is possible to leverage accessible and data rich domains, such as ground, to effectively develop marine DRL agents for use on Autonomous Surface Vehicle (ASV) navigation. This will allow rapid and iterative agent development without the risk of ASV loss, the cost and logistic overhead of marine deployment, and allow landlocked institutions to develop agents for marine applications.

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

Deep Reinforcement Learning Based Tracking Control of an Autonomous Surface Vessel in Natural Waters

Robust path following on rivers using bootstrapped reinforcement learning

Guidance and Control of Autonomous Surface Underwater Vehicles for Target Tracking in Ocean Environment by Deep Reinforcement Learning

AI on the Water: Applying DRL to Autonomous Vessel Navigation

Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment

2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following

Survey of Deep Learning for Autonomous Surface Vehicles in Marine Environments

Using Deep Reinforcement Learning Methods for Autonomous Vessels in 2D Environments

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Target Search Control Of Auv In Underwater Environment With Deep Reinforcement Learning

DRL-based Path Planning and Obstacle Avoidance of Autonomous Underwater Vehicle

Autonomous Surface Vehicle Control Method Using Deep Reinforcement Learning

Deep Reinforcement Learning for Vectored Thruster Autonomous Underwater Vehicle Control

Surface path tracking method of autonomous surface underwater vehicle based on deep reinforcement learning

Concise Deep Reinforcement Learning Obstacle Avoidance for Underactuated Unmanned Marine Vessels.

Aeolus Ocean -- A simulation environment for the autonomous COLREG-compliant navigation of Unmanned Surface Vehicles using Deep Reinforcement Learning and Maritime Object Detection