Abstract:Path planning is one of the key technologies to support the autonomous and safe navigation of unmanned surface vessel (USV). In order to solve the problems of slow convergence speed, many iterations, and unstable convergence results when the traditional reinforcement learning algorithm is applied to the path planning of the USV. In this paper, an improved hybrid path planning method based on Dyna architecture(IDH) is proposed. The Dyna architecture constructed based on model learning and direct reinforcement learning can give full play to the advantages of both and has good robustness in dealing with uncertain existing problems. The path searching method combining Q-learning and heuristic algorithm can learn quickly according to different environmental conditions and search for safe paths that meet the planning objectives (such as shortest path and lowest energy consumption, etc.). In this paper, the kinematic characteristics of target distance, distance to closest point of approach(DCPA), time to closest point of approach(TCPA), and steering angle are comprehensively considered when constructing the reward and punishment strategy and action selection rules of Q-Learning. Then the post smoother was developed to improve the performance of the generated route, reducing unnecessary ’jags’. Taking the actual environment maps as examples, the proposed planning method is simulated and verified. The Q-Learning, Dyna-Q and Rapid-exploration Random Tree(RRT) algorithm are chosen as the comparison method. Experiments show that the proposed improved path planning method has the best adaptability and robustness under the influence of actual environment maps. The performance of the proposed approach in term of total distance is superior over the other algorithm. In addition, the search speed is faster than the other two reinforcement learning algorithms.

Path Planning Method of Unmanned Surface Vessel Based on Strategy Integration

A Hybrid Path Planning Algorithm for Unmanned Surface Vehicles in Complex Environment with Dynamic Obstacles.

USV Application Scenario Expansion Based on Motion Control, Path Following and Velocity Planning

Optimal Path Planning of Unmanned Surface Vehicle under Current Environment

Achieving optimal-dynamic path planning for unmanned surface vehicles: A rational multi-objective approach and a sensory-vector re-planner

An Optimal-Path-Planning Method for Unmanned Surface Vehicles Based on a Novel Group Intelligence Algorithm

Efficient Path Planning Method of USV for Intelligent Target Search

Dynamic Path Planning Algorithm for Unmanned Surface Vehicle under Island-Reef Environment.

Path Planning Algorithm for Unmanned Surface Vessel Based on Multiobjective Reinforcement Learning

Path Planning of Mission-driven USVs Based on Stochastic Schemata Exploiter

A Hybrid Path-Planning Scheme for an Unmanned Surface Vehicle

Path planning for Multi-USV target coverage in complex environments

Multi-Behavior Fusion Based Potential Field Method for Path Planning of Unmanned Surface Vessel

Path Planning Method Based on D* lite Algorithm for Unmanned Surface Vehicles in Complex Environments

Physical Consistent Path Planning for Unmanned Surface Vehicles under Complex Marine Environment

Research on Path Planning Method of Unmanned Boat Based on Improved Artificial Potential Field Method

A Hybrid Intelligent Path Planning Method Based on Improved Dyna-H Architecture for Unmanned Surface Vessel

Dynamic Path Planning for Unmanned Surface Vessel Based on Improved Artificial Potential Field

Global-local hierarchical path planning scheme for unmanned surface vehicles under dynamically unforeseen environments

Path Planning for Unmanned Surface Vehicles Based on Modified Artificial Fish Swarm Algorithm with Local Optimizer

A new real‐time path planning for USV based on dynamic artificial potential field in complex environments