Abstract:This study proposes a framework for human-like autonomous car-following planning based on deep reinforcement learning (deep RL). Historical driving data are fed into a simulation environment where an RL agent learns from trial and error interactions based on a reward function that signals how much the agent deviates from the empirical data. Through these interactions, an optimal policy, or car-following model that maps in a human-like way from speed, relative speed between a lead and following vehicle, and inter-vehicle spacing to acceleration of a following vehicle is finally obtained. The model can be continuously updated when more data are fed in. Two thousand car-following periods extracted from the 2015 Shanghai Naturalistic Driving Study were used to train the model and compare its performance with that of traditional and recent data-driven car-following models. As shown by this study results, a deep deterministic policy gradient car-following model that uses disparity between simulated and observed speed as the reward function and considers a reaction delay of 1s, denoted as DDPGvRT, can reproduce human-like car-following behavior with higher accuracy than traditional and recent data-driven car-following models. Specifically, the DDPGvRT model has a spacing validation error of 18% and speed validation error of 5%, which are less than those of other models, including the intelligent driver model, models based on locally weighted regression, and conventional neural network-based models. Moreover, the DDPGvRT demonstrates good capability of generalization to various driving situations and can adapt to different drivers by continuously learning. This study demonstrates that reinforcement learning methodology can offer insight into driver behavior and can contribute to the development of human-like autonomous driving algorithms and traffic-flow models.

Hybrid-Driven Car-Following Model Based on Improved Composite Network and IDM

Car-Following Modeling Incorporating Driving Memory Based on Autoencoder and Long Short-Term Memory Neural Networks

Analysis of car–following behaviors based on data–driven and theory–driven car–following models: Heterogeneity and asymmetry

Composite Driver Car-Following Model Based on Neural Network Approach

Car-following model based on artificial potential field

MR-IDM -- Merge Reactive Intelligent Driver Model: Towards Enhancing Laterally Aware Car-following Models

An Intelligent Car-Following Model Based on Multi-Step Deep Reinforcement Learning

An Attention-Based Car-Following Model Based on Fused data

A Novel Car-Following Control Model Combining Machine Learning and Kinematics Models for Automated Vehicles

On the Impact of Prior Experiences in Car-Following Models: Model Development, Computational Efficiency, Comparative Analyses, and Extensive Applications

Driver Car-Following Behavior Modeling Using Neural Network Based on Real Traffic Experimental Data

A performance modeling and analysis of a novel vehicular traffic flow prediction system using a hybrid machine learning-based model

RACER: Rational Artificial Intelligence Car-following-model Enhanced by Reality

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

IDM-Follower: A Model-Informed Deep Learning Method for Long-Sequence Car-Following Trajectory Prediction

Modeling takeover time for non-driving related tasks using hybrid deep neural networks in conditionally automated vehicles

Adaptive Kalman-based hybrid car following strategy using TD3 and CACC

Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control

EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning

A bidirectional car-following model considering distance balance between adjacent vehicles