Abstract:Traffic scenarios in roundabouts pose substantial complexity for automated driving. Manually mapping all possible scenarios into a state space is labor-intensive and challenging. Deep reinforcement learning (DRL) with its ability to learn from interacting with the environment emerges as a promising solution for training such automated driving models. This study explores, employs, and implements various DRL algorithms, namely Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Trust Region Policy Optimization (TRPO) to instruct automated vehicles' driving through roundabouts. The driving state space, action space, and reward function are designed. The reward function considers safety, efficiency, comfort, and energy consumption to align with real-world requirements. All three tested DRL algorithms succeed in enabling automated vehicles to drive through the roundabout. To holistically evaluate the performance of these algorithms, this study establishes an evaluation methodology considering multiple indicators such as safety, efficiency, and comfort level. A method employing the Analytic Hierarchy Process is also developed to weigh these evaluation indicators. Experimental results on various testing scenarios reveal that the TRPO algorithm outperforms DDPG and PPO in terms of safety and efficiency, and PPO performs best in terms of comfort level. Lastly, to verify the model's adaptability and robustness regarding other driving scenarios, this study also deploys the model trained by TRPO to a range of different testing scenarios, e.g., highway driving and merging. Experimental results demonstrate that the TRPO model trained on only roundabout driving scenarios exhibits a certain degree of proficiency in highway driving and merging scenarios. This study provides a foundation for the application of automated driving with DRL in real traffic environments.

Transfer Learning-Based Deep Reinforcement Learning Approach for Robust Route Guidance in Mixed Traffic Environment

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

Constrained Reinforcement Learning for Vehicle Motion Planning with Topological Reachability Analysis

Transit Signal Priority Strategy with Heterogeneous Graph-Based Deep Reinforcement Learning for Autonomous Public Transit Vehicles

Self-Organized Routing for Autonomous Vehicles Via Deep Reinforcement Learning

Deep Reinforcement Learning for the Dynamic and Uncertain Vehicle Routing Problem

Comprehensive Training and Evaluation on Deep Reinforcement Learning for Automated Driving in Various Simulated Driving Maneuvers

Intelligent Cruise Guidance and Vehicle Resource Management With Deep Reinforcement Learning

Autonomous Highway Driving using Deep Reinforcement Learning

Longitudinal control of connected and automated vehicles among signalized intersections in mixed traffic flow with deep reinforcement learning approach

DRL4Route: A Deep Reinforcement Learning Framework for Pick-up and Delivery Route Prediction

Connected Automated Vehicle Cooperative Control with a Deep Reinforcement Learning Approach in a Mixed Traffic Environment

XRouting: Explainable Vehicle Rerouting for Urban Road Congestion Avoidance Using Deep Reinforcement Learning

Real-time deep reinforcement learning based vehicle navigation

RHRA-DRL: RSU-Assisted Hybrid Road-Aware Routing Using Distributed Reinforcement Learning in Internet of Vehicles

Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

DRL based platooning control with traffic signal synchronization for delay and fuel optimization

Performance Comparison of Deep RL Algorithms for Mixed Traffic Cooperative Lane-Changing

Transferable Deep Reinforcement Learning Framework for Autonomous Vehicles With Joint Radar-Data Communications

Safe, Efficient, Comfort, and Energy-saving Automated Driving through Roundabout Based on Deep Reinforcement Learning

A reinforcement learning based autonomous vehicle control in diverse daytime and weather scenarios