Abstract:The application of Deep Reinforcement Learning (DRL) has significantly impacted the development of autonomous driving technology in the field of intelligent transportation. However, in mixed traffic scenarios involving both human-driven vehicles (HDVs) and connected and autonomous vehicles (CAVs), challenges arise, particularly concerning information sharing and collaborative control among multiple intelligent agents using DRL. To address this issue, we propose a novel framework, namely Spatial-Temporal Deep Reinforcement Learning (ST-DRL), that enables collaborative control among multiple CAVs in mixed traffic scenarios. Initially, the traffic states involving multiple agents are constructed as graph-formatted data, which is then sequential created to represent continuous time intervals. With the data representation, interactive behaviors and dynamic characteristics among multiple intelligent agents are implicitly captured. Subsequently, to better represent the spatial relationships between vehicles, a graph enabling network is utilize to encode the vehicle states, which can contribute to the improvement of information sharing efficiency among multiple intelligent agents. Additionally, a spatial-temporal feature fusion network module is designed, which integrates graph convolutional networks (GCN) and gated recurrent units (GRU). It can effectively fuse independent spatial-temporal features and further enhance collaborative control performance. Through extensive experiments conducted in the SUMO traffic simulator and comparison with baseline methods, it is demonstrated that the ST-DRL framework achieves higher success rates in mixed traffic scenarios and exhibits better trade-offs between safety and efficiency. The analysis of the results indicates that ST-DRL has increased the success rate of the task by compared to the baseline method, while reducing model training and task completion times by respectively.

A Deep Reinforcement Learning-Based Approach for Autonomous Driving in Highway On-Ramp Merge

Autonomous Highway Merging in Mixed Traffic Using Reinforcement Learning and Motion Predictive Safety Controller

Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic

Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge

Learning an Efficient and Safe Policy for Highway Driving Using Supervised Learning and Reinforcement Learning.

Autonomous Ramp Merge Maneuver Based on Reinforcement Learning with Continuous Action Space

A Deep Reinforcement Learning Approach for Automated On-Ramp Merging

On-Ramp Merging for Highway Autonomous Driving: An Application of a New Safety Indicator in Deep Reinforcement Learning

Investigating Autonomous Vehicle Driving Strategies in Highway Ramp Merging Zones

Novel Decision-Making Strategy for Connected and Autonomous Vehicles in Highway On-Ramp Merging

Reinforcement Learning with Latent State Inference for Autonomous On-ramp Merging under Observation Delay

Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning

RACE-SM: Reinforcement Learning Based Autonomous Control for Social On-Ramp Merging

Combining Reinforcement Learning with Model Predictive Control for On-Ramp Merging

On Social Interactions of Merging Behaviors at Highway On-Ramps in Congested Traffic

Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Merging

A Systematic Study of Multi-Agent Deep Reinforcement Learning for Safe and Robust Autonomous Highway Ramp Entry

Graph-based multi agent reinforcement learning for on-ramp merging in mixed traffic

Multi-Vehicle Cooperative Decision-Making in Merging Area Based on Deep Multi-Agent Reinforcement Learning

Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment

Battery-aware cooperative merging strategy of connected electric vehicles based on reinforcement learning with hindsight experience replay