Abstract:Abstract This paper proposes a machine learning agent for automatically navigating a vessel in a confined channel subject to environmental conditions. The agent is trained and tested using a Ship Maneuvering Simulator and is responsible for commanding the rudder, so as to keep the vessel inside the channel with minimum distance from the center line, and to reach the final part of the channel with a prescribed thruster rotation level. The algorithm is based on deep reinforcement learning method and uses an efficient state-space representation. The advantage of using reinforcement learning is that it does not require any expert to directly teach the agent how to behave under particular conditions. The novelty of this work is that: it does not require previous knowledge on the vessel dynamic model and the maneuvering scenario; it is robust against fluctuations of environmental forces such as wind and current; it considers discrete actions of rudder commands emulating the pilot actions in a real maneuver. The developed method is convenient for simulations in scenarios or areas that were never navigated before, in which no previous navigation data can be used to train a conventional supervised learning agent. One direct application for this work is the integration with a realistic fast-time maneuvering simulator for new ports or operations. Both training and validation experiments focused on the unsheltered approach channel of the Suape Port, in Brazil; these experiments were run in a SMH-USP maneuvering simulator (real environmental conditions measured on-site were employed in simulations).

Port Channel Navigation Subjected to Environmental Conditions Using Reinforcement Learning