Abstract:Path planning is one of the key technologies to support the autonomous and safe navigation of unmanned surface vessel (USV). In order to solve the problems of slow convergence speed, many iterations, and unstable convergence results when the traditional reinforcement learning algorithm is applied to the path planning of the USV. In this paper, an improved hybrid path planning method based on Dyna architecture(IDH) is proposed. The Dyna architecture constructed based on model learning and direct reinforcement learning can give full play to the advantages of both and has good robustness in dealing with uncertain existing problems. The path searching method combining Q-learning and heuristic algorithm can learn quickly according to different environmental conditions and search for safe paths that meet the planning objectives (such as shortest path and lowest energy consumption, etc.). In this paper, the kinematic characteristics of target distance, distance to closest point of approach(DCPA), time to closest point of approach(TCPA), and steering angle are comprehensively considered when constructing the reward and punishment strategy and action selection rules of Q-Learning. Then the post smoother was developed to improve the performance of the generated route, reducing unnecessary ’jags’. Taking the actual environment maps as examples, the proposed planning method is simulated and verified. The Q-Learning, Dyna-Q and Rapid-exploration Random Tree(RRT) algorithm are chosen as the comparison method. Experiments show that the proposed improved path planning method has the best adaptability and robustness under the influence of actual environment maps. The performance of the proposed approach in term of total distance is superior over the other algorithm. In addition, the search speed is faster than the other two reinforcement learning algorithms.

A path planning approach for unmanned surface vehicles based on dynamic and fast Q-learning

Learning and Sampling-Based Informative Path Planning for AUVs in Ocean Current Fields

A Hybrid Path Planning Algorithm for Unmanned Surface Vehicles in Complex Environment with Dynamic Obstacles.

Path Planning of Unmanned Surface Vehicle Based on Improved Q-Learning Algorithm

Path Planning for Unmanned Surface Vehicle Based on Improved Q-Learning Algorithm

Optimal Path Planning of Unmanned Surface Vehicle under Current Environment

A Hybrid Intelligent Path Planning Method Based on Improved Dyna-H Architecture for Unmanned Surface Vessel

A Motion Camouflage-Inspired Path Planning Method for UAVs Based on Reinforcement Learning

Global path planning algorithm based on double DQN for multi-tasks amphibious unmanned surface vehicle

Local path planning for unmanned surface vehicle based on spatial and temporal sensing-enhanced deep Q-network

A Modified Q-learning Algorithm Facing for Path Planning and Obstacle Avoidance Method for Unmanned Aerial Vehicle

Path Planning Method Based on D* lite Algorithm for Unmanned Surface Vehicles in Complex Environments

Path Planning Technology of Unmanned Vehicle Based on Improved Deep Reinforcement Learning

An Algorithm of Complete Coverage Path Planning for Unmanned Surface Vehicle Based on Reinforcement Learning

A path planning strategy unified with a COLREGS collision avoidance function based on deep reinforcement learning and artificial potential field

USVs Path Planning for Maritime Search and Rescue Based on POS-DQN: Probability of Success-Deep Q-Network

An Improved Dueling Deep Double-Q Network Based on Prioritized Experience Replay for Path Planning of Unmanned Surface Vehicles

Path Planning of Unmanned Surface Vehicle Port Docking Based on Improved Double Deep Q-Network

Dynamic route planning method based on deep reinforcement learning and velocity obstacle

Dynamic Path Planning Algorithm for Unmanned Surface Vehicle under Island-Reef Environment.

Deep Reinforcement Learning-Based Path Planning of Underactuated Surface Vessels