Abstract:In the coming years and decades, autonomous vehicles (AVs) will become increasingly prevalent, offering new opportunities for safer and more convenient travel and potentially smarter traffic control methods exploiting automation and connectivity. Car following is a prime function in autonomous driving. Car following based on reinforcement learning has received attention in recent years with the goal of learning and achieving performance levels comparable to humans. However, most existing RL methods model car following as a unilateral problem, sensing only the vehicle ahead. Recent literature, however, Wang and Horn [16] has shown that bilateral car following that considers the vehicle ahead and the vehicle behind exhibits better system stability. In this paper we hypothesize that this bilateral car following can be learned using RL, while learning other goals such as efficiency maximisation, jerk minimization, and safety rewards leading to a learned model that outperforms human driving. We propose and introduce a Deep Reinforcement Learning (DRL) framework for car following control by integrating bilateral information into both state and reward function based on the bilateral control model (BCM) for car following control. Furthermore, we use a decentralized multi-agent reinforcement learning framework to generate the corresponding control action for each agent. Our simulation results demonstrate that our learned policy is better than the human driving policy in terms of (a) inter-vehicle headways, (b) average speed, (c) jerk, (d) Time to Collision (TTC) and (e) string stability.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: How to optimize car - following behavior through Deep Reinforcement Learning (DRL) to achieve more efficient, safer, and more comfortable driving performance than human driving, and improve the stability of the traffic system. Specifically, most of the existing reinforcement learning methods, when dealing with the car - following problem, usually only consider the information of the vehicle in front and model it as a unilateral problem. This limits the performance improvement of the car - following model. To improve this, this paper proposes two extensions: 1. **Optimize car - following performance**: Use deep reinforcement learning to optimize car - following behavior to achieve maximum efficiency, safety, and comfort. 2. **Integrate bilateral information**: Incorporate the information of both the front and rear vehicles into the state and reward functions simultaneously, inspired by the Bilateral Control Model (BCM). In addition, this paper also uses a decentralized multi - agent reinforcement learning framework to generate control actions for each agent. The experimental results show that the proposed strategy is superior to human - driving strategies in multiple aspects (such as inter - vehicle distance, average speed, acceleration change rate, Time to Collision (TTC), and string stability). ### Key point summary: - **Problem background**: The existing car - following models are mainly based on unilateral information, ignoring the influence of the rear vehicle, resulting in limited performance. - **Research objective**: By introducing bilateral information and deep reinforcement learning, improve the efficiency, safety, and comfort of car - following behavior, and at the same time improve the stability of the traffic system. - **Innovation points**: Propose a deep reinforcement learning framework combined with bilateral information to optimize car - following behavior, and further improve performance through a multi - agent learning framework. Through these improvements, this paper aims to develop an intelligent car - following system that can outperform human - driving performance, thereby improving the overall traffic situation.

Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Reinforcement Learning-Based Approach to Reduce Velocity Error in Car-Following for Autonomous Connected Vehicles

Improved Deep Reinforcement Learning for Car-Following Decision-Making

An Intelligent Car-Following Model Based on Multi-Step Deep Reinforcement Learning

Autonomous Highway Driving using Deep Reinforcement Learning

Learning to Drive Like Human Beings: A Method Based on Deep Reinforcement Learning

Towards robust car-following based on deep reinforcement learning

Longitudinal control of automated vehicles: A novel approach by integrating deep reinforcement learning with intelligent driver model

Multi-Level Objective Control of AVs at a Saturated Signalized Intersection with Multi-Agent Deep Reinforcement Learning Approach

SECRM-2D: RL-Based Efficient and Comfortable Route-Following Autonomous Driving with Analytic Safety Guarantees

Deep Reinforcement Learning Car-Following Model Considering Longitudinal and Lateral Control

Task-Driven Autonomous Driving: Balanced Strategies Integrating Curriculum Reinforcement Learning and Residual Policy

Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving

Reinforcement Learning-Based High-Speed Path Following Control for Autonomous Vehicles

Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning

Modified DDPG car-following model with a real-world human driving experience with CARLA simulator

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Dynamic Target Following Control for Autonomous Vehicles with Deep Reinforcement Learning.

Joint Optimization of Sensing, Decision-making and Motion-controlling for Autonomous Vehicles: A Deep Reinforcement Learning Approach