Abstract:Deep deterministic policy gradient (DDPG)-based car-following strategy can break through the constraints of the differential equation model due to the ability of exploration on complex environments. However, the car-following performance of DDPG is usually degraded by unreasonable reward function design, insufficient training, and low sampling efficiency. In order to solve this kind of problem, a hybrid car-following strategy based on DDPG and cooperative adaptive cruise control (CACC) is proposed. First, the car-following process is modeled as the Markov decision process to calculate CACC and DDPG simultaneously at each frame. Given a current state, two actions are obtained from CACC and DDPG, respectively. Then, an optimal action, corresponding to the one offering a larger reward, is chosen as the output of the hybrid strategy. Meanwhile, a rule is designed to ensure that the change rate of acceleration is smaller than the desired value. Therefore, the proposed strategy not only guarantees the basic performance of car-following through CACC but also makes full use of the advantages of exploration on complex environments via DDPG. Finally, simulation results show that the car-following performance of the proposed strategy is improved compared with that of DDPG and CACC. Note to Practitioners—This article presents a new car-following strategy, which avoids the impact of deep deterministic policy gradient (DDPG) performance degradation on the system. In the proposed strategy, DDPG is replaced with cooperative adaptive cruise control (CACC) when the performance of DDPG is worse than that of CACC. Meanwhile, a switching rule is designed to guarantee that the change rate of acceleration is smaller than the threshold. Simulation results show that the performance of hybrid car-following strategy has been improved compared with that of only using CACC or DDPG. Moreover, the proposed strategy has the advantages of low computational burden, high real-time-performance, and good scalability.

A Data-Driven Behavior Generation Algorithm In Car-Following Scenarios

A Shared Control Approach for Autonomous Vehicles via Driver Behaviors Learning

Car-following Behavior Model Learning Using Timed Automata

A Study on Learning and Simulating Personalized Car-Following Driving Style

An Adaptive Cruise Control System Based on Self-Learning Algorithm for Driver Characteristics

A Predictive Control Algorithm Based on Driving Behavior Data Model.

A generative car-following model conditioned on driving styles

Hybrid Car-Following Strategy Based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control

Model Predictive Adaptive Cruise Control of Intelligent Electric Vehicles Based on Deep Reinforcement Learning Algorithm FWOR Driver Characteristics

Two-mode adaptive cruise control design with humans' driving habits consideration

Personalized Adaptive Cruise Control with Deep Reinforcement Learning

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

A Data-Based Approach to Path Following Controller Design for Autonomous Vehicles

Car-following model based on artificial potential field

Real-Time Determination of Driver'S Driving Behavior During Car Following

A New Hybrid Model Predictive Controller Design for Adaptive Cruise of Autonomous Electric Vehicles

Car-Following Models: A Multidisciplinary Review

Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving

A Systematic Solution of Human Driving Behavior Modeling and Simulation for Automated Vehicle Studies

EditFollower: Tunable Car Following Models for Customizable Adaptive Cruise Control Systems