Abstract:Reinforcement learning (RL)–based car-following (CF) control strategies have attracted significant attention in academia, emerging as a prominent research topic in recent years. Most of these control strategies focus solely on the motion status of the immediately preceding vehicle. However, with the development of vehicle-to-vehicle (V2V) communication technologies, intelligent vehicles such as connected autonomous vehicles (CAVs) can gather information about surrounding vehicles. Therefore, this study proposes an RL-based CF control strategy that takes multivehicle scenarios into account. First, the trajectories of two preceding vehicles and one following vehicle relative to the subject vehicle (SV) are extracted from a highD dataset to construct the environment. Then the twin-delayed deep deterministic policy gradient (TD3) algorithm is implemented as the control strategy for the agent. Furthermore, a sequence-to-sequence (seq2seq) module is developed to predict the uncertain motion statuses of surrounding vehicles. Once integrated into the RL framework, this module enables the agent to account for dynamic changes in the traffic environment, enhancing its robustness. Finally, the performance of the CF control strategy is validated both in the highD dataset and in two traffic perturbation scenarios. In the highD dataset, the TD3-based prediction CF control strategy outperforms standard RL algorithms in terms of convergence speed and rewards. Its performance also surpasses that of human drivers in safety, efficiency, comfort, and fuel consumption. In traffic perturbation scenarios, the performance of the proposed CF control strategy is compared with the model predictive controller (MPC). The results show that the TD3-based prediction CF control strategy effectively mitigates undesired traffic waves caused by the perturbations from the head vehicle. Simultaneously, it maintains the desired traffic state and consistently ensures a stable and efficient traffic flow.

Feedback Forecasting Based Deep Deterministic Policy Gradient Algorithm for Car-Following of Autonomous Vehicle

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Combining Deep Reinforcement Learning and Safety Based Control for Autonomous Driving

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

Deep Deterministic Policy Gradient Algorithm Based on Convolutional Block Attention for Autonomous Driving.

Optimization of Safety and Comfort in Car-following Scene Based on Reinforcement Learning

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Soft Collision Avoidance Based Car Following Algorithm for Autonomous Driving with Reinforcement Learning

Autonomous Vehicles Roundup Strategy by Reinforcement Learning with Prediction Trajectory

Vehicle-Following Control Based on Deep Reinforcement Learning

Obstacle Avoidance for Self-Driving Vehicle with Reinforcement Learning

Autonomous Driving Control Using the DDPG and RDPG Algorithms

Improved Deep Reinforcement Learning for Car-Following Decision-Making

Deep Reinforcement Learning Car-Following Model Considering Longitudinal and Lateral Control

Deep Reinforcement Learning for Advanced Longitudinal Control and Collision Avoidance in High-Risk Driving Scenarios

An Automatic Driving Control Method Based on Deep Deterministic Policy Gradient

A Novel Vehicle Platoon Following Controller Based on Deep Deterministic Policy Gradient Algorithms

Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control

Velocity Control in Car-Following Behavior with Autonomous Vehicles Using Reinforcement Learning

Dynamic Target Following Control for Autonomous Vehicles with Deep Reinforcement Learning.