Abstract:Path planning is one of the key technologies to support the autonomous and safe navigation of unmanned surface vessel (USV). In order to solve the problems of slow convergence speed, many iterations, and unstable convergence results when the traditional reinforcement learning algorithm is applied to the path planning of the USV. In this paper, an improved hybrid path planning method based on Dyna architecture(IDH) is proposed. The Dyna architecture constructed based on model learning and direct reinforcement learning can give full play to the advantages of both and has good robustness in dealing with uncertain existing problems. The path searching method combining Q-learning and heuristic algorithm can learn quickly according to different environmental conditions and search for safe paths that meet the planning objectives (such as shortest path and lowest energy consumption, etc.). In this paper, the kinematic characteristics of target distance, distance to closest point of approach(DCPA), time to closest point of approach(TCPA), and steering angle are comprehensively considered when constructing the reward and punishment strategy and action selection rules of Q-Learning. Then the post smoother was developed to improve the performance of the generated route, reducing unnecessary ’jags’. Taking the actual environment maps as examples, the proposed planning method is simulated and verified. The Q-Learning, Dyna-Q and Rapid-exploration Random Tree(RRT) algorithm are chosen as the comparison method. Experiments show that the proposed improved path planning method has the best adaptability and robustness under the influence of actual environment maps. The performance of the proposed approach in term of total distance is superior over the other algorithm. In addition, the search speed is faster than the other two reinforcement learning algorithms.

Path Planning of Unmanned Surface Vehicle Port Docking Based on Improved Double Deep Q-Network

Path Planning of Autonomous Underwater Vehicle Departing from Port Based on Improved Deep Q-Network

An Improved Dueling Deep Double-Q Network Based on Prioritized Experience Replay for Path Planning of Unmanned Surface Vehicles

Global path planning algorithm based on double DQN for multi-tasks amphibious unmanned surface vehicle

Path Planning Technology of Unmanned Vehicle Based on Improved Deep Reinforcement Learning

A Hybrid Path Planning Algorithm for Unmanned Surface Vehicles in Complex Environment with Dynamic Obstacles.

A Hybrid Intelligent Path Planning Method Based on Improved Dyna-H Architecture for Unmanned Surface Vessel

A path planning approach for unmanned surface vehicles based on dynamic and fast Q-learning

An Improved DQN Algorithm for Automated Guided Vehicle Pathfinding Problem in Port Environment

Path Planning of Unmanned Surface Vehicle Based on Improved Q-Learning Algorithm

Path Planning for Unmanned Surface Vehicle Based on Improved Q-Learning Algorithm

Deep Reinforcement Learning-Based Path Planning of Underactuated Surface Vessels

Path Planning Method of Unmanned Surface Vessel Based on Strategy Integration

Local path planning for unmanned surface vehicle based on spatial and temporal sensing-enhanced deep Q-network

Autonomous docking trajectory optimization for unmanned surface vehicle: A hierarchical method

An Improved Quantum-Behaved Particle Swarm Optimization Algorithm Combined with Reinforcement Learning for AUV Path Planning

Path Planning Algorithm for Unmanned Surface Vessel Based on Multiobjective Reinforcement Learning

USVs Path Planning for Maritime Search and Rescue Based on POS-DQN: Probability of Success-Deep Q-Network

Path Planning Method for AUV Docking Based on Adaptive Quantum-Behaved Particle Swarm Optimization

A path planning algorithm for unmanned surface vessel with pose constraints in an unknown environment

Path Planning Method Based on D* lite Algorithm for Unmanned Surface Vehicles in Complex Environments