Abstract:In autonomous driving, behavioral decision-making and trajectory planning remain huge challenges due to the large amount of uncertainty in environments and complex interaction relationships between the ego vehicle and other traffic participants. In this paper, we propose a novel fixed-horizon constrained reinforcement learning (RL) framework to solve decision-making and planning problems. Firstly, to introduce lane-level global navigation information into the lane state representation and avoid constant lane changes, we propose the constrained A-star algorithm, which can get the optimal path without constant lane changes. The optimality of the algorithm is also theoretically guaranteed. Then, to balance safety, comfort, and goal completion (reaching targets), we construct the planning problem as a constrained RL problem, in which the reward function is designed for goal completion, and two fixed-horizon constraints are developed for safety and comfort, respectively. Subsequently, a motion planning policy network (planner) with vectorized input is constructed. Finally, a dual ascent optimization method is proposed to train the planner network. With the advantage of being able to fully explore in the environment, the agent can learn an efficient decision-making and planning policy. In addition, benefiting from modeling the safety and comfort of the ego vehicle as constraints, the learned policy can guarantee the safety of the ego vehicle and achieve a good balance between goal completion and comfort. Experiments demonstrate that the proposed algorithm can achieve superior performance than existing rule-based, imitation learning-based, and typical RL-based methods.

Three-Dimensional Autonomous Entry Trajectory Planning Via Hybrid Action Reinforcement Learning

Real-time adaptive entry trajectory generation with modular policy and deep reinforcement learning

Online Trajectory Planning Method for Midcourse Guidance Phase Based on Deep Reinforcement Learning

A RDA-Based Deep Reinforcement Learning Approach for Autonomous Motion Planning of UAV in Dynamic Unknown Environments

Multi-UAV Adaptive Cooperative Formation Trajectory Planning Based on an Improved MATD3 Algorithm of Deep Reinforcement Learning

Real-time Trajectory Planning for Hypersonic Entry Flight Via Curriculum Reinforcement Learning

TD3 Based Collision Free Motion Planning for Robot Navigation

Trajectory Planning for Airborne Radar in Extended Target Tracking Based on Deep Reinforcement Learning

A Hybrid Human-in-the-Loop Deep Reinforcement Learning Method for UAV Motion Planning for Long Trajectories with Unpredictable Obstacles

Deep Reinforcement Learning-Based 3D Trajectory Planning for Cellular Connected UAV

Deep Reinforcement Learning Based Trajectory Planning Under Uncertain Constraints

A Reinforcement Learning Based Motion Planner for Quadrotor Autonomous Flight in Dense Environment

Trajectory Planning with Deep Reinforcement Learning in High-Level Action Spaces

Motion Planner with Fixed-Horizon Constrained Reinforcement Learning for Complex Autonomous Driving Scenarios

DRL-Based Trajectory Tracking for Motion-Related Modules in Autonomous Driving

Autonomous Gliding Entry Guidance with Geographic Constraints

Reinforcement Learning Based Trajectory Planning for Autonomous Vehicles

Autonomous localized path planning algorithm for UAVs based on TD3 strategy

Autonomous Navigation of UAV in Multi-Obstacle Environments Based on a Deep Reinforcement Learning Approach

Entry trajectory planning based on three-dimensional acceleration profile guidance

Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learning