Abstract:Reinforcement learning (RL)–based car-following (CF) control strategies have attracted significant attention in academia, emerging as a prominent research topic in recent years. Most of these control strategies focus solely on the motion status of the immediately preceding vehicle. However, with the development of vehicle-to-vehicle (V2V) communication technologies, intelligent vehicles such as connected autonomous vehicles (CAVs) can gather information about surrounding vehicles. Therefore, this study proposes an RL-based CF control strategy that takes multivehicle scenarios into account. First, the trajectories of two preceding vehicles and one following vehicle relative to the subject vehicle (SV) are extracted from a highD dataset to construct the environment. Then the twin-delayed deep deterministic policy gradient (TD3) algorithm is implemented as the control strategy for the agent. Furthermore, a sequence-to-sequence (seq2seq) module is developed to predict the uncertain motion statuses of surrounding vehicles. Once integrated into the RL framework, this module enables the agent to account for dynamic changes in the traffic environment, enhancing its robustness. Finally, the performance of the CF control strategy is validated both in the highD dataset and in two traffic perturbation scenarios. In the highD dataset, the TD3-based prediction CF control strategy outperforms standard RL algorithms in terms of convergence speed and rewards. Its performance also surpasses that of human drivers in safety, efficiency, comfort, and fuel consumption. In traffic perturbation scenarios, the performance of the proposed CF control strategy is compared with the model predictive controller (MPC). The results show that the TD3-based prediction CF control strategy effectively mitigates undesired traffic waves caused by the perturbations from the head vehicle. Simultaneously, it maintains the desired traffic state and consistently ensures a stable and efficient traffic flow.

Path Following with Supervised Deep Reinforcement Learning

Learning an Efficient and Safe Policy for Highway Driving Using Supervised Learning and Reinforcement Learning.

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

Reinforcement Learning-Based High-Speed Path Following Control for Autonomous Vehicles

Path tracking control based on Deep reinforcement learning in Autonomous driving

Path Following for Autonomous Mobile Robots with Deep Reinforcement Learning

Path Following Control for Unmanned Surface Vehicles: A Reinforcement Learning-Based Method with Experimental Validation.

Learning to Drive Like Human Beings: A Method Based on Deep Reinforcement Learning

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

Dynamic Target Following Control for Autonomous Vehicles with Deep Reinforcement Learning.

ReinforcementDriving: Exploring Trajectories and Navigation for Autonomous Vehicles

Combining Deep Reinforcement Learning and Safety Based Control for Autonomous Driving

A Data-Based Approach to Path Following Controller Design for Autonomous Vehicles

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Intelligent Vehicle Decision-Making and Trajectory Planning Method Based on Deep Reinforcement Learning in the Frenet Space

Research into Autonomous Vehicles Following and Obstacle Avoidance Based on Deep Reinforcement Learning Method under Map Constraints

Joint Optimization of Sensing, Decision-making and Motion-controlling for Autonomous Vehicles: A Deep Reinforcement Learning Approach

Car-following Method Based on Inverse Reinforcement Learning for Autonomous Vehicle Decision-Making

Deep Reinforcement Learning Car-Following Model Considering Longitudinal and Lateral Control