Abstract:Traffic Signal Control (TSC) plays a crucial role in mitigating congestion. The extensive integration of Deep Reinforcement Learning (DRL) into TSC, fueled by the advancements in artificial intelligence, has yielded remarkable efficacy. Nevertheless, prevailing DRL models are typically developed within homogeneous road networks, characterized by identical intersection types and relatively balanced traffic flow patterns. This predominant approach often results in an oversight of the impact of intersection heterogeneity, encompassing variations in intersection geometric structure, traffic demands, signal timing design, and other pertinent factors, on action selections by researchers. This paper addresses the aforementioned gap by introducing a value-decomposition-based spatiotemporal graph attention multi-agent DRL model (MARL_SGAT) that expressly accommodates intersection heterogeneity. Specifically, a heterogeneous correlation index is designed using road network structural parameters and serves to formulate a heterogeneous reward function, enabling the quantification of action execution improvements in diverse road network environments. To address the intricate spatiotemporal features of traffic flows within the heterogeneous road network, the model strategically selects and incorporates these features during the formulation by leveraging a spatiotemporal graph attention network. On these bases, double dual networks are introduced to convert the individual-global-max constraint of action selection into a value range constraint of the action advantage function, thereby facilitating model learning. Simulation results showcase the superior performance of the MARL_SGAT algorithm when compared to seven baseline algorithms. Specifically, the algorithm manifests in reduced average vehicle delays, decreased number of stops, and elevated travel speeds within the controlled road network. These advantages are particularly conspicuous in heterogeneous road network environments.

CTRL: Cooperative Traffic Tolling Via Reinforcement Learning

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

MAGT-toll: A multi-agent reinforcement learning approach to dynamic traffic congestion pricing

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Dynamic Tolling in Arc-based Traffic Assignment Models

Cooperative Deep Reinforcement Learning for Large-Scale Traffic Grid Signal Control

Cooperative Reinforcement Learning on Traffic Signal Control

Deep Reinforcement Learning Algorithm for Dynamic Pricing of Express Lanes with Multiple Access Locations

Optimizing Traffic Signal Control in Mixed Traffic Scenarios: A Predictive Traffic Information-Based Deep Reinforcement Learning Approach

Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction

Diagnosing Reinforcement Learning for Traffic Signal Control

Learning and managing stochastic network traffic dynamics: an iterative and interactive approach

Optimizing Traffic Efficiency Via a Reinforcement Learning Approach Based on Time Allocation

Enhanced Delta-tolling: Traffic Optimization via Policy Gradient Reinforcement Learning

Carbon Dioxide Emission Reduction-Oriented Optimal Control of Traffic Signals in Mixed Traffic Flow Based on Deep Reinforcement Learning

Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implementation

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

Multi-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach