Abstract:Reinforcement learning (RL)–based car-following (CF) control strategies have attracted significant attention in academia, emerging as a prominent research topic in recent years. Most of these control strategies focus solely on the motion status of the immediately preceding vehicle. However, with the development of vehicle-to-vehicle (V2V) communication technologies, intelligent vehicles such as connected autonomous vehicles (CAVs) can gather information about surrounding vehicles. Therefore, this study proposes an RL-based CF control strategy that takes multivehicle scenarios into account. First, the trajectories of two preceding vehicles and one following vehicle relative to the subject vehicle (SV) are extracted from a highD dataset to construct the environment. Then the twin-delayed deep deterministic policy gradient (TD3) algorithm is implemented as the control strategy for the agent. Furthermore, a sequence-to-sequence (seq2seq) module is developed to predict the uncertain motion statuses of surrounding vehicles. Once integrated into the RL framework, this module enables the agent to account for dynamic changes in the traffic environment, enhancing its robustness. Finally, the performance of the CF control strategy is validated both in the highD dataset and in two traffic perturbation scenarios. In the highD dataset, the TD3-based prediction CF control strategy outperforms standard RL algorithms in terms of convergence speed and rewards. Its performance also surpasses that of human drivers in safety, efficiency, comfort, and fuel consumption. In traffic perturbation scenarios, the performance of the proposed CF control strategy is compared with the model predictive controller (MPC). The results show that the TD3-based prediction CF control strategy effectively mitigates undesired traffic waves caused by the perturbations from the head vehicle. Simultaneously, it maintains the desired traffic state and consistently ensures a stable and efficient traffic flow.

Hybrid Deep Learning for Dynamic Total Transfer Capability Control

Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle

A Deep Reinforcement Learning Based Hierarchical Eco-Driving Strategy for Connected and Automated HEVs

A Deep Reinforcement Learning-Based Real-Time Control for Transfer Limits of Critical Inter-Corridors.

Nonparametric Probabilistic Total Transfer Capability Estimation with Deep Learning

Deep Reinforcement Learning-Based Multi-Objective Control of Hybrid Power System Combined with Road Recognition under Time-Varying Environment

Data-driven transferred energy management strategy for hybrid electric vehicles via deep reinforcement learning

Learning-Aided Optimal Power Flow Based Fast Total Transfer Capability Calculation

Intelligent Control Strategy for Transient Response of a Variable Geometry Turbocharger System Based on Deep Reinforcement Learning

Dynamic Target Following Control for Autonomous Vehicles with Deep Reinforcement Learning.

Path tracking control based on Deep reinforcement learning in Autonomous driving

Analytic Deep Learning-based Surrogate Model for Operational Planning with Dynamic TTC Constraints

Longitudinal control of automated vehicles: A novel approach by integrating deep reinforcement learning with intelligent driver model

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

End-to-End Autonomous Driving Decision-Making Solution Based on Pri-TD3

Surrogate‐assisted Optimal Re‐dispatch Control for Risk‐aware Regulation of Dynamic Total Transfer Capability

Autonomous Driving Control Using the DDPG and RDPG Algorithms

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

A Transferable Energy Management Strategy for Hybrid Electric Vehicles Via Dueling Deep Deterministic Policy Gradient