Abstract:Rule-based traditional motion planning methods usually perform well with prior knowledge of the macro-scale environments but encounter challenges in unknown and uncertain environments. Deep reinforcement learning (DRL) is a solution that can effectively deal with micro-scale unknown and uncertain environments. Nevertheless, DRL is unstable and lacks interpretability. Therefore, it raises a new challenge: how to combine the effectiveness and overcome the drawbacks of the two methods while guaranteeing stability in uncertain environments. In this study, a multi-constraint and multi-scale motion planning method is proposed for automated driving with the use of constrained reinforcement learning (RL), named RLTT, and comprising RL, a topological reachability analysis used for vehicle path space (TPS), and a trajectory lane model (TLM). First, a dynamic model of vehicles is formulated; then, TLM is developed on the basis of the dynamic model, thus constraining RL action and state space. Second, macro-scale path planning is achieved through TPS, and in the micro-scale range, discrete routing points are achieved via RLTT. Third, the proposed motion planning method is designed by combining sophisticated rules, and a theoretical analysis is provided to guarantee the efficiency of our method. Finally, related experiments are conducted to evaluate the effectiveness of the proposed method; our method can reduce 19.9% of the distance cost in the experiments as compared to the traditional method. Experimental results indicate that the proposed method can help mitigate the gap between data-driven and traditional methods, provide better performance for automated driving, and facilitate the use of RL methods in more fields.

Motion Planning Using Reinforcement Learning Method for Underactuated Ship Berthing.

Constrained Reinforcement Learning for Vehicle Motion Planning with Topological Reachability Analysis

An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning

A RDA-Based Deep Reinforcement Learning Approach for Autonomous Motion Planning of UAV in Dynamic Unknown Environments

Path Planning Technology of Unmanned Vehicle Based on Improved Deep Reinforcement Learning

Reinforcement Learning from Demonstrations by Novel Interactive Expert and Application to Automatic Berthing Control Systems for Unmanned Surface Vessel

A Novel Path Planning Approach For Unmanned Ships Based On Deep Reinforcement Learning

Neural-network-based Reinforcement Learning Control for Path Following of Underactuated Ships

Deep Reinforcement Learning Based Path Planning and Collision Avoidance for Smart Ships in Complex Environments

Reinforcement Learning Based Bi-stage Control for Semi-enclosed Berth

Path Planning of Unmanned Underwater Vehicles Based on Deep Reinforcement Learning Algorithm

Path Planning for Underactuated Unmanned Surface Vehicle Swarm Based on Deep Reinforcement Learning

Soft Actor-Critic and Risk Assessment-Based Reinforcement Learning Method for Ship Path Planning

Research and Design of an Autonomous Underwater Vehicle Path Planning Method Based on Deep Reinforcement Learning

Deep Reinforcement Learning-Based Path Planning of Underactuated Surface Vessels

Intelligent Ship Decision System Based on DDPG Algorithm

TD3 Based Collision Free Motion Planning for Robot Navigation

Path Planning of Maritime Autonomous Surface Ships in Unknown Environment with Reinforcement Learning

An Obstacle-Avoidance Motion Planning Method for Redundant Space Robot via Reinforcement Learning

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning