Abstract:This study proposes autonomous eco-driving strategies for a traffic environment with limited information available based on three popular Reinforcement Learning (RL) algorithms for continuous actions, i.e., Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO) and Soft Actor–Critic (SAC) approaches, to address a serious challenge in the literature. The challenge is, despite the potential of connected and automated vehicles (CAVs) to diminish traffic interruptions caused by traffic signals on urban streets, for a long period of time, the accurate and immediate exchange of information via vehicle-to-everything (V2X) communication remains a problem because of the low penetration rate of CAVs and communication quality issues. The available information is assumed to only include signal phase and timing (SPaT) information and traffic states of the leading vehicle, respectively. To optimize the overall driving performance, a trade-off among safety, efficiency, energy, and ride comfort is considered in the reward function. Moreover, a hybrid policy is proposed to take advantage of RL and an analytical car-following model. The strategies enable CAVs to safely, efficiently and comfortably traverse signalized intersections while all the other vehicles are unconnected human-driven vehicles (HVs). Trajectories from the pNEUMA dataset are used to train and test the proposed models. The performance of the proposed models is compared to the naturalistic driving data, the Intelligent Driver Model (IDM) and an eco-driving method based on rules and optimization (Trigo). Testing results show that DDPG and SAC with the hybrid policy (HybridSAC) have the best overall performance, i.e., better than human drivers in all aspects; better safety, energy and comfort than the Trigo model; similar performance to the IDM but better energy efficiency. The temporal and spatial generalization capabilities of the RL-based methods are also tested, among which HybridSAC has the best performance as a whole.

Multi-Objective Optimization of Vehicle-Following Control for Connected Electric Vehicles Based on Deep Deterministic Policy Gradient

A Safety-Enhanced Eco-Driving Strategy for Connected and Autonomous Vehicles: A Hierarchical and Distributed Framework

A Deep Reinforcement Learning Based Hierarchical Eco-Driving Strategy for Connected and Automated HEVs

Collaborative Optimization of Energy Management Strategy and Adaptive Cruise Control Based on Deep Reinforcement Learning

Multi-objective Eco-Driving Strategy for Connected and Automated Electric Vehicles Considering Complex Urban Traffic Influence Factors

Deep reinforcement learning and reward shaping based eco-driving control for automated HEVs among signalized intersections

Ecological Driving Framework of Hybrid Electric Vehicle Based on Heterogeneous Multi Agent Deep Reinforcement Learning

Overcoming driving challenges in complex urban traffic: A multi-objective eco-driving strategy via safety model based reinforcement learning

Longitudinal autonomous driving based on game theory for intelligent hybrid electric vehicles with connectivity

Hybrid Car-Following Strategy Based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control

Multi-objective Optimization for Connected and Automated Vehicles Using Machine Learning and Model Predictive Control

Advanced deep deterministic policy gradient based energy management strategy design for dual-motor four-wheel-drive electric vehicle

Navigating Electric Vehicles Along a Signalized Corridor via Reinforcement Learning: Toward Adaptive Eco-Driving Control

Ecological Cruising Control of Connected Electric Vehicle: a Deep Reinforcement Learning Approach

Multi-objective optimization of safety, comfort and economy of hybrid electric vehicle in car-following scenario

Multi-agent reinforcement learning for ecological car-following control in mixed traffic

Stochastic Co-Optimization of Speed Planning and Powertrain Control with Dynamic Probabilistic Constraints for Safe and Ecological Driving

A multi-objective reinforcement learning-based velocity optimization approach for electric trucks considering battery degradation mitigation

A Two-Layer Real-Time Optimal Control for Intelligent Hybrid Electric Vehicles with Connectivity.

Eco-driving strategies using reinforcement learning for mixed traffic in the vicinity of signalized intersections