Abstract:Decision-making is an important component of autonomous driving perception, decision-making, planning, and control pipeline, which undertakes the task of how the ego vehicle makes high-level decision-making behaviors (such as lane change and car following) after sensing the environmental state, and then these high-level decision-making behaviors can be transmitted to the downstream planning and control module for specific low-level action execution. Based on the method of deep reinforcement learning (specifically, Deep Q network (DQN) and its variants), an integrated lateral and longitudinal decision-making model for autonomous driving is proposed in a multilane highway environment with both autonomous driving vehicle (ADV) and manual driving vehicle (MDV). The classic MOBIL and IDM models are used for the lateral and longitudinal decisions of MDV (i.e., lane changing and car following), while the lateral and longitudinal decisions of ADV are dominated by deep reinforcement learning models. In addition, this paper also uses the nonlinear kinematic bicycle model and two-point visual control model to realize the low-level control of both MDV and ADV. By setting a reasonable state, action, and reward function, this paper has carried out a large number of simulation experiments on the proposed autonomous driving decision-making model based on deep reinforcement learning in a three-lane road environment. The results show that under such scenario setting conditions, the deep reinforcement learning-based model proposed in this paper performs well in autonomous driving safety and travel efficiency. At the same time, when compared with the classical rule-based decision-making model (MOBIL&IDM), it is found that the model proposed in this paper can significantly achieve better results in episode rewards after stable training. In addition, through a large number of hyper-parameter tuning experiments, the performance of DQN, DDQN, and dueling DQN models, which are also deep reinforcement learning-based decision-making models, under different hyper-parametric configurations is compared and analyzed, which can provide a valuable reference for the specific scenario application of these models.

Deep Reinforcement Learning Based Decision-Making Strategy of Autonomous Vehicle in Highway Uncertain Driving Environments

Learning an Efficient and Safe Policy for Highway Driving Using Supervised Learning and Reinforcement Learning.

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

Decision-making Strategy on Highway for Autonomous Vehicles using Deep Reinforcement Learning

Risk-aware Deep Reinforcement Learning for Decision-making and Planning of Autonomous Vehicles

Continuous Decision-Making in Lane Changing and Overtaking Maneuvers for Unmanned Vehicles: A Risk-Aware Reinforcement Learning Approach with Task Decomposition

Decision Making of Autonomous Vehicles in Lane Change Scenarios: Deep Reinforcement Learning Approaches with Risk Awareness

Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving

Deep Q-Network Based Decision Making for Autonomous Driving

An Integrated Lateral and Longitudinal Decision-Making Model for Autonomous Driving Based on Deep Reinforcement Learning

Dueling Deep Q Network for Highway Decision Making in Autonomous Vehicles: A Case Study

Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment

Research on decision-making of lane-changing of automated vehicles in highway confluence area based on deep reinforcement learning

Rule-based Reinforcement Learning for Lane Change Decision-making: A Risk Assessment Approach

Towards Robust Decision-Making for Autonomous Driving on Highway

Autonomous Driving in the Uncertain Traffic——a Deep Reinforcement Learning Approach

Safe Decision Controller for Autonomous DrivingBased on Deep Reinforcement Learning inNondeterministic Environment

Autonomous Highway Driving using Deep Reinforcement Learning

Tactical Driving Decisions of Unmanned Ground Vehicles in Complex Highway Environments: A Deep Reinforcement Learning Approach

Mandatory Lane-Changing Decision-Making in Dense Traffic for Autonomous Vehicles Based on Deep Reinforcement Learning

A Safe and Efficient Lane Change Decision-Making Strategy of Autonomous Driving Based on Deep Reinforcement Learning