Abstract:This study proposes autonomous eco-driving strategies for a traffic environment with limited information available based on three popular Reinforcement Learning (RL) algorithms for continuous actions, i.e., Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO) and Soft Actor–Critic (SAC) approaches, to address a serious challenge in the literature. The challenge is, despite the potential of connected and automated vehicles (CAVs) to diminish traffic interruptions caused by traffic signals on urban streets, for a long period of time, the accurate and immediate exchange of information via vehicle-to-everything (V2X) communication remains a problem because of the low penetration rate of CAVs and communication quality issues. The available information is assumed to only include signal phase and timing (SPaT) information and traffic states of the leading vehicle, respectively. To optimize the overall driving performance, a trade-off among safety, efficiency, energy, and ride comfort is considered in the reward function. Moreover, a hybrid policy is proposed to take advantage of RL and an analytical car-following model. The strategies enable CAVs to safely, efficiently and comfortably traverse signalized intersections while all the other vehicles are unconnected human-driven vehicles (HVs). Trajectories from the pNEUMA dataset are used to train and test the proposed models. The performance of the proposed models is compared to the naturalistic driving data, the Intelligent Driver Model (IDM) and an eco-driving method based on rules and optimization (Trigo). Testing results show that DDPG and SAC with the hybrid policy (HybridSAC) have the best overall performance, i.e., better than human drivers in all aspects; better safety, energy and comfort than the Trigo model; similar performance to the IDM but better energy efficiency. The temporal and spatial generalization capabilities of the RL-based methods are also tested, among which HybridSAC has the best performance as a whole.

An efficient planning method based on deep reinforcement learning with hybrid actions for autonomous driving on highway

An End-to-end Deep Reinforcement Learning Approach for the Long-term Short-term Planning on the Frenet Space

Highway Decision-Making and Motion Planning for Autonomous Driving via Soft Actor-Critic

Decision-making for Autonomous Vehicles on Highway: Deep Reinforcement Learning with Continuous Action Horizon

Autonomous Highway Merging in Mixed Traffic Using Reinforcement Learning and Motion Predictive Safety Controller

An Improved Deep Reinforcement Learning Algorithm for Path Planning in Unmanned Driving

Let Hybrid A* Path Planner Obey Traffic Rules: A Deep Reinforcement Learning-Based Planning Framework

Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Driving

Autonomous Highway Driving using Deep Reinforcement Learning

A Hybrid Deep Reinforcement Learning and Optimal Control Architecture for Autonomous Highway Driving

Decision-making Strategy on Highway for Autonomous Vehicles using Deep Reinforcement Learning

Eco-driving strategies using reinforcement learning for mixed traffic in the vicinity of signalized intersections

A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning

A Safe and Efficient Lane Change Decision-Making Strategy of Autonomous Driving Based on Deep Reinforcement Learning

Integration of Reinforcement Learning Based Behavior Planning With Sampling Based Motion Planning for Automated Driving

Path Planning for Autonomous Vehicles in Unknown Dynamic Environment Based on Deep Reinforcement Learning

Intelligent Vehicle Decision-Making and Trajectory Planning Method Based on Deep Reinforcement Learning in the Frenet Space

A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance

Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learning

A Real-Time Intelligent Speed Optimization Planner Using Reinforcement Learning

An End-to-End Path Planner Combining Potential Field Method With Deep Reinforcement Learning