Abstract:Decision-making is an important component of autonomous driving perception, decision-making, planning, and control pipeline, which undertakes the task of how the ego vehicle makes high-level decision-making behaviors (such as lane change and car following) after sensing the environmental state, and then these high-level decision-making behaviors can be transmitted to the downstream planning and control module for specific low-level action execution. Based on the method of deep reinforcement learning (specifically, Deep Q network (DQN) and its variants), an integrated lateral and longitudinal decision-making model for autonomous driving is proposed in a multilane highway environment with both autonomous driving vehicle (ADV) and manual driving vehicle (MDV). The classic MOBIL and IDM models are used for the lateral and longitudinal decisions of MDV (i.e., lane changing and car following), while the lateral and longitudinal decisions of ADV are dominated by deep reinforcement learning models. In addition, this paper also uses the nonlinear kinematic bicycle model and two-point visual control model to realize the low-level control of both MDV and ADV. By setting a reasonable state, action, and reward function, this paper has carried out a large number of simulation experiments on the proposed autonomous driving decision-making model based on deep reinforcement learning in a three-lane road environment. The results show that under such scenario setting conditions, the deep reinforcement learning-based model proposed in this paper performs well in autonomous driving safety and travel efficiency. At the same time, when compared with the classical rule-based decision-making model (MOBIL&IDM), it is found that the model proposed in this paper can significantly achieve better results in episode rewards after stable training. In addition, through a large number of hyper-parameter tuning experiments, the performance of DQN, DDQN, and dueling DQN models, which are also deep reinforcement learning-based decision-making models, under different hyper-parametric configurations is compared and analyzed, which can provide a valuable reference for the specific scenario application of these models.

An Intelligent Car-Following Model Based on Multi-Step Deep Reinforcement Learning

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

Improved Deep Reinforcement Learning for Car-Following Decision-Making

Optimization of Safety and Comfort in Car-following Scene Based on Reinforcement Learning

A Deep Reinforcement Learning Based Car Following Model for Electric Vehicle

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Autonomous Driving in the Uncertain Traffic——a Deep Reinforcement Learning Approach

An Integrated Lateral and Longitudinal Decision-Making Model for Autonomous Driving Based on Deep Reinforcement Learning

Autonomous Motion Decision-making based on Deep Reinforcement Learning for Autonomous Driving

Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model

Car-following Behavior Modeling with Maximum Entropy Deep Inverse Reinforcement Learning

Enhancing Car-Following Performance in Traffic Oscillations Using Expert Demonstration Reinforcement Learning

Car-following Method Based on Inverse Reinforcement Learning for Autonomous Vehicle Decision-Making

Comprehensive Training and Evaluation on Deep Reinforcement Learning for Automated Driving in Various Simulated Driving Maneuvers

A Hierarchical Framework for Multi-Lane Autonomous Driving Based on Reinforcement Learning

Development of an Efficient Driving Strategy for Connected and Automated Vehicles at Signalized Intersections: A Reinforcement Learning Approach

A Decision-Making Strategy for Car Following Based on Naturalist Driving Data via Deep Reinforcement Learning

A Multi-sensing Input and Multi-constraint Reward Mechanism Based Deep Reinforcement Learning Method for Self-driving Policy Learning

Formulation and validation of a car-following model based on deep reinforcement learning

From Naturalistic Traffic Data to Learning-Based Driving Policy: A Sim-to-Real Study

An Integrated Model for Autonomous Speed and Lane Change Decision-Making Based on Deep Reinforcement Learning.