Abstract:This study proposes autonomous eco-driving strategies for a traffic environment with limited information available based on three popular Reinforcement Learning (RL) algorithms for continuous actions, i.e., Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO) and Soft Actor–Critic (SAC) approaches, to address a serious challenge in the literature. The challenge is, despite the potential of connected and automated vehicles (CAVs) to diminish traffic interruptions caused by traffic signals on urban streets, for a long period of time, the accurate and immediate exchange of information via vehicle-to-everything (V2X) communication remains a problem because of the low penetration rate of CAVs and communication quality issues. The available information is assumed to only include signal phase and timing (SPaT) information and traffic states of the leading vehicle, respectively. To optimize the overall driving performance, a trade-off among safety, efficiency, energy, and ride comfort is considered in the reward function. Moreover, a hybrid policy is proposed to take advantage of RL and an analytical car-following model. The strategies enable CAVs to safely, efficiently and comfortably traverse signalized intersections while all the other vehicles are unconnected human-driven vehicles (HVs). Trajectories from the pNEUMA dataset are used to train and test the proposed models. The performance of the proposed models is compared to the naturalistic driving data, the Intelligent Driver Model (IDM) and an eco-driving method based on rules and optimization (Trigo). Testing results show that DDPG and SAC with the hybrid policy (HybridSAC) have the best overall performance, i.e., better than human drivers in all aspects; better safety, energy and comfort than the Trigo model; similar performance to the IDM but better energy efficiency. The temporal and spatial generalization capabilities of the RL-based methods are also tested, among which HybridSAC has the best performance as a whole.

Safe Efficient Policy Optimization Algorithm for Unsignalized Intersection Navigation

Reinforcement Learning Aided Sequential Optimization for Unsignalized Intersection Management of Robot Traffic

Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions

Eco-driving strategies using reinforcement learning for mixed traffic in the vicinity of signalized intersections

Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimization

Courteous Behavior of Automated Vehicles at Unsignalized Intersections Via Reinforcement Learning

A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment

Autonomous navigation at unsignalized intersections: A coupled reinforcement learning and model predictive control approach

Roadside Units Assisted Localized Automated Vehicle Maneuvering: An Offline Reinforcement Learning Approach

A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections

Proximal Policy Optimization Through a Deep Reinforcement Learning Framework for Multiple Autonomous Vehicles at a Non-Signalized Intersection

Hybrid Reinforcement Learning-Based Eco-Driving Strategy for Connected and Automated Vehicles at Signalized Intersections

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Reward-Driven Automated Curriculum Learning for Interaction-Aware Self-Driving at Unsignalized Intersections

Self-Learned Autonomous Driving at Unsignalized Intersections: A Hierarchical Reinforced Learning Approach for Feasible Decision-Making

Safe Deep Q-Network for Autonomous Vehicles at Unsignalized Intersection

Navigating Occluded Intersections with Autonomous Vehicles using Deep Reinforcement Learning

Eco-driving at signalized intersections: a parameterized reinforcement learning approach

Self-Learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersections

Eco-driving for Electric Connected Vehicles at Signalized Intersections: A Parameterized Reinforcement Learning approach