Abstract:Path planning is one of the key technologies to support the autonomous and safe navigation of unmanned surface vessel (USV). In order to solve the problems of slow convergence speed, many iterations, and unstable convergence results when the traditional reinforcement learning algorithm is applied to the path planning of the USV. In this paper, an improved hybrid path planning method based on Dyna architecture(IDH) is proposed. The Dyna architecture constructed based on model learning and direct reinforcement learning can give full play to the advantages of both and has good robustness in dealing with uncertain existing problems. The path searching method combining Q-learning and heuristic algorithm can learn quickly according to different environmental conditions and search for safe paths that meet the planning objectives (such as shortest path and lowest energy consumption, etc.). In this paper, the kinematic characteristics of target distance, distance to closest point of approach(DCPA), time to closest point of approach(TCPA), and steering angle are comprehensively considered when constructing the reward and punishment strategy and action selection rules of Q-Learning. Then the post smoother was developed to improve the performance of the generated route, reducing unnecessary ’jags’. Taking the actual environment maps as examples, the proposed planning method is simulated and verified. The Q-Learning, Dyna-Q and Rapid-exploration Random Tree(RRT) algorithm are chosen as the comparison method. Experiments show that the proposed improved path planning method has the best adaptability and robustness under the influence of actual environment maps. The performance of the proposed approach in term of total distance is superior over the other algorithm. In addition, the search speed is faster than the other two reinforcement learning algorithms.

Path Planning for Unmanned Marine Vessels Based on Improved Reinforcement Learning

A Novel Path Planning Approach For Unmanned Ships Based On Deep Reinforcement Learning

Deep Reinforcement Learning-Based Path Control and Optimization for Unmanned Ships

Path Planning Technology of Unmanned Vehicle Based on Improved Deep Reinforcement Learning

Path Planning of Unmanned Surface Vehicle Based on Improved Q-Learning Algorithm

An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning

Path Planning Algorithm for Unmanned Surface Vessel Based on Multiobjective Reinforcement Learning

Research on Path Planning Method of Unmanned Ship under Complex Sea Conditions

Path Planning of Maritime Autonomous Surface Ships in Unknown Environment with Reinforcement Learning

A Hybrid Intelligent Path Planning Method Based on Improved Dyna-H Architecture for Unmanned Surface Vessel

A Path Planning Method Based on Deep Reinforcement Learning for AUV in Complex Marine Environment

A knowledge-free path planning approach for smart ships based on reinforcement learning

AUV Path Planning with Kinematic Constraints in Unknown Environment Using Reinforcement Learning.

Research Status and Development Trend of Path Planning Algorithm for Unmanned Vehicles

Improved reinforcement learning for collision-free local path planning of dynamic obstacle

Application of Improved RRT Algorithm in Unmanned Surface Vehicle Path Planning

Research on Path Planning based on Unmanned Ship

Path Planning of Unmanned Underwater Vehicles Based on Deep Reinforcement Learning Algorithm

A path planning strategy unified with a COLREGS collision avoidance function based on deep reinforcement learning and artificial potential field

Coverage path planning for maritime search and rescue using reinforcement learning

An Algorithm of Complete Coverage Path Planning for Unmanned Surface Vehicle Based on Reinforcement Learning