Abstract:Rule-based traditional motion planning methods usually perform well with prior knowledge of the macro-scale environments but encounter challenges in unknown and uncertain environments. Deep reinforcement learning (DRL) is a solution that can effectively deal with micro-scale unknown and uncertain environments. Nevertheless, DRL is unstable and lacks interpretability. Therefore, it raises a new challenge: how to combine the effectiveness and overcome the drawbacks of the two methods while guaranteeing stability in uncertain environments. In this study, a multi-constraint and multi-scale motion planning method is proposed for automated driving with the use of constrained reinforcement learning (RL), named RLTT, and comprising RL, a topological reachability analysis used for vehicle path space (TPS), and a trajectory lane model (TLM). First, a dynamic model of vehicles is formulated; then, TLM is developed on the basis of the dynamic model, thus constraining RL action and state space. Second, macro-scale path planning is achieved through TPS, and in the micro-scale range, discrete routing points are achieved via RLTT. Third, the proposed motion planning method is designed by combining sophisticated rules, and a theoretical analysis is provided to guarantee the efficiency of our method. Finally, related experiments are conducted to evaluate the effectiveness of the proposed method; our method can reduce 19.9% of the distance cost in the experiments as compared to the traditional method. Experimental results indicate that the proposed method can help mitigate the gap between data-driven and traditional methods, provide better performance for automated driving, and facilitate the use of RL methods in more fields.

Motion Planning for Autonomous Vehicles in Uncertain Environments Using Hierarchical Distributional Reinforcement Learning

Optimization-based Motion Planning for Autonomous Parking Considering Dynamic Obstacle: A Hierarchical Framework

Rollout-Based Interactive Motion Planning for Automated Vehicles *

Learning Hierarchical Behavior and Motion Planning for Autonomous Driving.

Behavior and Interaction-aware Motion Planning for Autonomous Driving Vehicles based on Hierarchical Intention and Motion Prediction

Hierarchical Reinforcement Learning for Autonomous Decision Making and Motion Planning of Intelligent Vehicles

Constrained Reinforcement Learning for Vehicle Motion Planning with Topological Reachability Analysis

A Hierarchical Motion Planning System for Driving in Changing Environments: Framework, Algorithms, and Verifications

Navigating Autonomous Vehicles in Uncertain Environments with Distributional Reinforcement Learning

Safety-field Based Motion Planning for Proactive Autonomous Driving in Dynamic Environment

Towards Safe Motion Planning for Autonomous Driving in Highway

Hierarchical Motion Planning System with Consideration of the Dynamic Lane-Changing Behaviour

Conditional DQN-Based Motion Planning With Fuzzy Logic for Autonomous Driving

Hierarchical Motion Planning for Autonomous Driving in Large-Scale Complex Scenarios

Hierarchical Motion Planning and Tracking for Autonomous Vehicles Using Global Heuristic Based Potential Field and Reinforcement Learning Based Predictive Control

A Motion Planning Method for Autonomous Vehicles

Real-time local path planning strategy based on deep distributional reinforcement learning

Receding-Horizon Reinforcement Learning Approach for Kinodynamic Motion Planning of Autonomous Vehicles

Reinforcement Learning Based Trajectory Planning for Autonomous Vehicles

Real-time Motion Planning for autonomous vehicles in dynamic environments

A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning