Abstract:The application of Deep Reinforcement Learning (DRL) has significantly impacted the development of autonomous driving technology in the field of intelligent transportation. However, in mixed traffic scenarios involving both human-driven vehicles (HDVs) and connected and autonomous vehicles (CAVs), challenges arise, particularly concerning information sharing and collaborative control among multiple intelligent agents using DRL. To address this issue, we propose a novel framework, namely Spatial-Temporal Deep Reinforcement Learning (ST-DRL), that enables collaborative control among multiple CAVs in mixed traffic scenarios. Initially, the traffic states involving multiple agents are constructed as graph-formatted data, which is then sequential created to represent continuous time intervals. With the data representation, interactive behaviors and dynamic characteristics among multiple intelligent agents are implicitly captured. Subsequently, to better represent the spatial relationships between vehicles, a graph enabling network is utilize to encode the vehicle states, which can contribute to the improvement of information sharing efficiency among multiple intelligent agents. Additionally, a spatial-temporal feature fusion network module is designed, which integrates graph convolutional networks (GCN) and gated recurrent units (GRU). It can effectively fuse independent spatial-temporal features and further enhance collaborative control performance. Through extensive experiments conducted in the SUMO traffic simulator and comparison with baseline methods, it is demonstrated that the ST-DRL framework achieves higher success rates in mixed traffic scenarios and exhibits better trade-offs between safety and efficiency. The analysis of the results indicates that ST-DRL has increased the success rate of the task by compared to the baseline method, while reducing model training and task completion times by respectively.

Real-time deep reinforcement learning based vehicle navigation

Reinforcement Learning for Vehicle Route Optimization in SUMO

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

SoC-VRP: A Deep-Reinforcement-Learning-Based Vehicle Route Planning Mechanism for Service-Oriented Cooperative ITS

A Hybrid of Deep Reinforcement Learning and Local Search for the Vehicle Routing Problems

Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning

Population Game-Assisted Multi-Agent Reinforcement Learning Method for Dynamic Multi-Vehicle Route Selection

Joint Optimization of Traffic Signal Control and Vehicle Routing in Signalized Road Networks using Multi-Agent Deep Reinforcement Learning

Self-Organized Routing for Autonomous Vehicles Via Deep Reinforcement Learning

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Hierarchical Reinforcement Learning for Dynamic Autonomous Vehicle Navigation at Intelligent Intersections

Deep Reinforcement Learning for the Dynamic and Uncertain Vehicle Routing Problem

Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network

Cooperative Optimization of Traffic Signals and Vehicle Speed Using a Novel Multi-agent Deep Reinforcement Learning

Intelligent Vehicle Decision-Making and Trajectory Planning Method Based on Deep Reinforcement Learning in the Frenet Space

Using a Deep Reinforcement Learning Agent for Traffic Signal Control

A Multi-Objective Optimal Control Method for Navigating Connected and Automated Vehicles at Signalized Intersections Based on Reinforcement Learning

Deep reinforcement learning navigation via decision transformer in autonomous driving

Graph-based multi agent reinforcement learning for on-ramp merging in mixed traffic

A multi‐agent deep reinforcement learning approach for traffic signal coordination

Book Your Green Wave: Exploiting Navigation Information for Intelligent Traffic Signal Control