Abstract:This study proposes autonomous eco-driving strategies for a traffic environment with limited information available based on three popular Reinforcement Learning (RL) algorithms for continuous actions, i.e., Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO) and Soft Actor–Critic (SAC) approaches, to address a serious challenge in the literature. The challenge is, despite the potential of connected and automated vehicles (CAVs) to diminish traffic interruptions caused by traffic signals on urban streets, for a long period of time, the accurate and immediate exchange of information via vehicle-to-everything (V2X) communication remains a problem because of the low penetration rate of CAVs and communication quality issues. The available information is assumed to only include signal phase and timing (SPaT) information and traffic states of the leading vehicle, respectively. To optimize the overall driving performance, a trade-off among safety, efficiency, energy, and ride comfort is considered in the reward function. Moreover, a hybrid policy is proposed to take advantage of RL and an analytical car-following model. The strategies enable CAVs to safely, efficiently and comfortably traverse signalized intersections while all the other vehicles are unconnected human-driven vehicles (HVs). Trajectories from the pNEUMA dataset are used to train and test the proposed models. The performance of the proposed models is compared to the naturalistic driving data, the Intelligent Driver Model (IDM) and an eco-driving method based on rules and optimization (Trigo). Testing results show that DDPG and SAC with the hybrid policy (HybridSAC) have the best overall performance, i.e., better than human drivers in all aspects; better safety, energy and comfort than the Trigo model; similar performance to the IDM but better energy efficiency. The temporal and spatial generalization capabilities of the RL-based methods are also tested, among which HybridSAC has the best performance as a whole.

Anti-Disturbance Self-Supervised Reinforcement Learning for Perturbed Car-Following System

Reinforcement Learning-Based Approach to Reduce Velocity Error in Car-Following for Autonomous Connected Vehicles

Enhancing Car-Following Performance in Traffic Oscillations Using Expert Demonstration Reinforcement Learning

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Adaptive robust control algorithm for enhanced path-tracking performance of automated driving in critical scenarios

Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

Perceptual Interaction-Based Path Tracking Control of Autonomous Vehicles under DoS Attacks: A Reinforcement Learning Approach

Deep Reinforcement Learning Car-Following Model Considering Longitudinal and Lateral Control

Eco-driving strategies using reinforcement learning for mixed traffic in the vicinity of signalized intersections

A Bounded Rationality-Aware Car-Following Strategy for Alleviating Cut-In Events and Traffic Disturbances in Traffic Oscillations

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Vehicle Extreme Control Based on Offline Reinforcement Leaning

A deep reinforcement learning-based active suspension control algorithm considering deterministic experience tracing for autonomous vehicle

Safe Reinforcement Learning-Based Eco-Driving Control for Mixed Traffic Flows With Disturbances

Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Control

Hybrid car following control for CAVs: Integrating linear feedback and deep reinforcement learning to stabilize mixed traffic

Trustworthy autonomous driving via defense-aware robust reinforcement learning against worst-case observational perturbations

Improved Robustness and Safety for Autonomous Vehicle Control with Adversarial Reinforcement Learning

SECRM-2D: RL-Based Efficient and Comfortable Route-Following Autonomous Driving with Analytic Safety Guarantees