Abstract:The application of Deep Reinforcement Learning (DRL) has significantly impacted the development of autonomous driving technology in the field of intelligent transportation. However, in mixed traffic scenarios involving both human-driven vehicles (HDVs) and connected and autonomous vehicles (CAVs), challenges arise, particularly concerning information sharing and collaborative control among multiple intelligent agents using DRL. To address this issue, we propose a novel framework, namely Spatial-Temporal Deep Reinforcement Learning (ST-DRL), that enables collaborative control among multiple CAVs in mixed traffic scenarios. Initially, the traffic states involving multiple agents are constructed as graph-formatted data, which is then sequential created to represent continuous time intervals. With the data representation, interactive behaviors and dynamic characteristics among multiple intelligent agents are implicitly captured. Subsequently, to better represent the spatial relationships between vehicles, a graph enabling network is utilize to encode the vehicle states, which can contribute to the improvement of information sharing efficiency among multiple intelligent agents. Additionally, a spatial-temporal feature fusion network module is designed, which integrates graph convolutional networks (GCN) and gated recurrent units (GRU). It can effectively fuse independent spatial-temporal features and further enhance collaborative control performance. Through extensive experiments conducted in the SUMO traffic simulator and comparison with baseline methods, it is demonstrated that the ST-DRL framework achieves higher success rates in mixed traffic scenarios and exhibits better trade-offs between safety and efficiency. The analysis of the results indicates that ST-DRL has increased the success rate of the task by compared to the baseline method, while reducing model training and task completion times by respectively.

Autonomous On-ramp Merge Strategy Using Deep Reinforcement Learning in Uncertain Highway Environment

Autonomous Highway Merging in Mixed Traffic Using Reinforcement Learning and Motion Predictive Safety Controller

On-Ramp Merging for Highway Autonomous Driving: An Application of a New Safety Indicator in Deep Reinforcement Learning

A Deep Reinforcement Learning-Based Approach for Autonomous Driving in Highway On-Ramp Merge

Investigating Autonomous Vehicle Driving Strategies in Highway Ramp Merging Zones

Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge

A Deep Reinforcement Learning Approach for Automated On-Ramp Merging

Autonomous Ramp Merge Maneuver Based on Reinforcement Learning with Continuous Action Space

Graph-based multi agent reinforcement learning for on-ramp merging in mixed traffic

Combining Reinforcement Learning with Model Predictive Control for On-Ramp Merging

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

Reinforcement Learning with Latent State Inference for Autonomous On-ramp Merging under Observation Delay

Decision-making Strategy on Highway for Autonomous Vehicles using Deep Reinforcement Learning

Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning

Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic

A Highway Entrance Ramp Control Method Based on Deep Reinforcement Learning

Ramp Merging Strategy Based on Dual-layer Optimization Model in Non-connected Scenarios

Novel Decision-Making Strategy for Connected and Autonomous Vehicles in Highway On-Ramp Merging

Poster: Reliable On-Ramp Merging Via Multimodal Reinforcement Learning

Leveraging on Deep Reinforcement Learning for Autonomous Safe Decision-Making in Highway On-ramp Merging (Student Abstract)

Continuous Decision-Making in Lane Changing and Overtaking Maneuvers for Unmanned Vehicles: A Risk-Aware Reinforcement Learning Approach with Task Decomposition