Abstract:Reinforcement learning (RL) methods are commonly regarded as effective solutions for designing intelligent driving policies. Nonetheless, even if the RL policy is converged after training, it is notoriously difficult to ensure safety. In particular, RL policy is susceptible to insecurity in the presence of long-tail or unseen traffic scenarios, i.e., out-of-distribution test data. Therefore, the design of the RL-based decision-making method must account for this shift in distribution. This paper proposes a robust decision-making framework for autonomous driving on the highway to improve driving safety. First, a Deep Deterministic Policy Gradient (DDPG)-based RL policy that directly maps observations to actions is constructed. Subsequently, the model uncertainty of the DDPG policy is evaluated at runtime to quantify the policy's reliability and identify unseen scenarios. In addition, a complementary principle-based policy is developed using the intelligent driver model (IDM) and the model for minimizing overall braking induced by lane changes (MOBIL). It will take over the DDPG policy when encountering unseen scenarios to guarantee a lower-bound performance of the decision-making system. Finally, the proposed method is implemented on an embedded system, i.e., NVIDIA Jetson AGX Xavier, and out-of-training distribution challenging cases are considered in the experiment, i.e., observation with sensor noise, traffic density increasing significantly, objects falling from the front vehicle, and road construction causing temporal changes in road structure. Results indicate that the proposed framework outperforms state-of-the-art benchmarks. Additionally, the code is provided https://github.com/Kayne0401/Robust-Decision-Making-Framework.

Distributional Soft Actor-Critic for Decision-Making in On-Ramp Merge Scenarios

Decision-Making under On-Ramp merge Scenarios by Distributional Soft Actor-Critic Algorithm

Research on robust decision making for intelligent connected vehicle at highway on-ramp

Autonomous Highway Merging in Mixed Traffic Using Reinforcement Learning and Motion Predictive Safety Controller

Encoding Distributional Soft Actor-Critic for Autonomous Driving in Multi-lane Scenarios

Applications of Distributional Soft Actor-Critic in Real-world Autonomous Driving

A Discrete Soft Actor-Critic Decision-Making Strategy With Sample Filter for Freeway Autonomous Driving

On-Ramp Merging for Highway Autonomous Driving: An Application of a New Safety Indicator in Deep Reinforcement Learning

Enhancing Cooperation of Vehicle Merging Control in Heavy Traffic Using Communication-Based Soft Actor-Critic Algorithm

Discretionary Lane-Change Decision and Control via Parameterized Soft Actor-Critic for Hybrid Action Space

Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for Discretionary Lane Change

Social-Aware Decision Algorithm for On-ramp Merging Based on Level-k Gaming

Discretionary Lane-Change Decision and Control via Parameterized Soft Actor–Critic for Hybrid Action Space

Highway Decision-Making and Motion Planning for Autonomous Driving via Soft Actor-Critic

Towards Robust Decision-Making for Autonomous Highway Driving Based on Safe Reinforcement Learning

Towards Robust Decision-Making for Autonomous Driving on Highway

Deep Reinforcement Learning Based Decision-Making Strategy of Autonomous Vehicle in Highway Uncertain Driving Environments

A Safe and Efficient Lane Change Decision-Making Strategy of Autonomous Driving Based on Deep Reinforcement Learning

Autonomous Ramp Merge Maneuver Based on Reinforcement Learning with Continuous Action Space

Novel Decision-Making Strategy for Connected and Autonomous Vehicles in Highway On-Ramp Merging

Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge