Abstract:In this paper, an adversarial deep reinforcement learning-based control method is proposed to address the issue of robust depth tracking of an underactuated autonomous underwater vehicle in the presence of intrinsic coupled dynamics and external disturbances. First, long-short-term-memory neural network is presented to memorize and predict the changes in the state of vehicle, and a cascaded multilayer perception projects the output into action space of vehicle. Subsequently, adversarial deep reinforcement learning scheme is applied to the training of control agent by introducing an adversary which counteracts the control behavior, whereby the agent is enabled to learn the control strategy in different distributions of state transition. For evaluation of the performance, a control agent is pre-trained in simulation environment based on the reliable digital model of a real vehicle, and the simulation environment is paced at one iteration per second to align with real-time operations to ensure the portability of training result. Furthermore, the training cost is also extremely reduced. Finally, experiments are conducted with time-varying disturbances to further prove the feasibility of the proposed learning-based control scheme on a prototype of underwater vehicle in towing tank. Moreover, comparative experiment results show the better robustness performance of the learning-based control agent than that of classic line-of-sight based proportional–integral–derivative and adaptive line-of-sight based proportional–integral–derivative controllers in different scenarios.

Autonomous Reinforcement Control of Underwater Vehicles Based on Monocular Depth Vision

Autonomous Reinforcement Control of Visual Underwater Vehicles: Real-Time Experiments Using Computer Vision

End-To-End Sensorimotor Control Problems Of Auvs With Deep Reinforcement Learning

Adversarial Deep Reinforcement Learning Based Robust Depth Tracking Control for Underactuated Autonomous Underwater Vehicle

Depth Control of Autonomous Underwater Vehicle Using Deep Reinforcement Learning

Guidance and Control of Autonomous Surface Underwater Vehicles for Target Tracking in Ocean Environment by Deep Reinforcement Learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Autonomous Underwater Pipeline Tracking Control Based on Visual Images.

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

End-to-End Deep Reinforcement Learning for Image-Based UAV Autonomous Control

Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning

Deep Reinforcement Learning Based Optimal Trajectory Tracking Control of Autonomous Underwater Vehicle

A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances.

Vision-based Environment Perception and Autonomous Obstacle Avoidance for Unmanned Underwater Vehicle

Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle

Three-Dimensional Path-Following Control of an Autonomous Underwater Vehicle Based on Deep Reinforcement Learning

Adaptive Formation Motion Planning and Control of Autonomous Underwater Vehicles Using Deep Reinforcement Learning

Depth Control of Model-Free AUVs Via Reinforcement Learning

Monocular vision guided deep reinforcement learning UAV systems with representation learning perception

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning