Abstract:Deep deterministic policy gradient (DDPG)-based car-following strategy can break through the constraints of the differential equation model due to the ability of exploration on complex environments. However, the car-following performance of DDPG is usually degraded by unreasonable reward function design, insufficient training, and low sampling efficiency. In order to solve this kind of problem, a hybrid car-following strategy based on DDPG and cooperative adaptive cruise control (CACC) is proposed. First, the car-following process is modeled as the Markov decision process to calculate CACC and DDPG simultaneously at each frame. Given a current state, two actions are obtained from CACC and DDPG, respectively. Then, an optimal action, corresponding to the one offering a larger reward, is chosen as the output of the hybrid strategy. Meanwhile, a rule is designed to ensure that the change rate of acceleration is smaller than the desired value. Therefore, the proposed strategy not only guarantees the basic performance of car-following through CACC but also makes full use of the advantages of exploration on complex environments via DDPG. Finally, simulation results show that the car-following performance of the proposed strategy is improved compared with that of DDPG and CACC. Note to Practitioners—This article presents a new car-following strategy, which avoids the impact of deep deterministic policy gradient (DDPG) performance degradation on the system. In the proposed strategy, DDPG is replaced with cooperative adaptive cruise control (CACC) when the performance of DDPG is worse than that of CACC. Meanwhile, a switching rule is designed to guarantee that the change rate of acceleration is smaller than the threshold. Simulation results show that the performance of hybrid car-following strategy has been improved compared with that of only using CACC or DDPG. Moreover, the proposed strategy has the advantages of low computational burden, high real-time-performance, and good scalability.

Car‐following Strategy of Intelligent Connected Vehicle Using Extended Disturbance Observer Adjusted by Reinforcement Learning

Reinforcement Learning-Based Approach to Reduce Velocity Error in Car-Following for Autonomous Connected Vehicles

Anti-Disturbance Self-Supervised Reinforcement Learning for Perturbed Car-Following System

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Hybrid car following control for CAVs: Integrating linear feedback and deep reinforcement learning to stabilize mixed traffic

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Adaptive Model Predictive Control-Based Path Following Control for Four-Wheel Independent Drive Automated Vehicles

Enhancing Car-Following Performance in Traffic Oscillations Using Expert Demonstration Reinforcement Learning

Model Predictive Control-Based Eco-Driving Strategy for CAV

High-Accuracy, High-Efficiency, and Comfortable Car-Following Strategy Based on TD3 for Wide-to-Narrow Road Sections

Attentive Hybrid Reinforcement Learning-Based Eco-Driving Strategy for Connected Vehicles with Hybrid Action Spaces and Surrounding Vehicles Attention

Hybrid Car-Following Strategy Based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control

Navigating Electric Vehicles Along a Signalized Corridor via Reinforcement Learning: Toward Adaptive Eco-Driving Control

A New Vehicle Path-Following Strategy of the Steering Driver Model Using General Predictive Control Method

Model Predictive Adaptive Cruise Control of Intelligent Electric Vehicles Based on Deep Reinforcement Learning Algorithm FWOR Driver Characteristics

An enhanced eco-driving strategy based on reinforcement learning for connected electric vehicles: cooperative velocity and lane-changing control

An Adaptive Constrained Path Following Control Scheme for Autonomous Electric Vehicles.

Learning-Based Predictive Control for Vehicle Following Problems

Design and Simulation of Car-Following Control System for Intelligent Vehicles

Distributed Connected Automated Vehicles Control under Real‐Time Aggregated Macroscopic Car Following Behavior Estimation Based on Deep Reinforcement Learning