Abstract:Traffic Signal Control (TSC) plays a crucial role in mitigating congestion. The extensive integration of Deep Reinforcement Learning (DRL) into TSC, fueled by the advancements in artificial intelligence, has yielded remarkable efficacy. Nevertheless, prevailing DRL models are typically developed within homogeneous road networks, characterized by identical intersection types and relatively balanced traffic flow patterns. This predominant approach often results in an oversight of the impact of intersection heterogeneity, encompassing variations in intersection geometric structure, traffic demands, signal timing design, and other pertinent factors, on action selections by researchers. This paper addresses the aforementioned gap by introducing a value-decomposition-based spatiotemporal graph attention multi-agent DRL model (MARL_SGAT) that expressly accommodates intersection heterogeneity. Specifically, a heterogeneous correlation index is designed using road network structural parameters and serves to formulate a heterogeneous reward function, enabling the quantification of action execution improvements in diverse road network environments. To address the intricate spatiotemporal features of traffic flows within the heterogeneous road network, the model strategically selects and incorporates these features during the formulation by leveraging a spatiotemporal graph attention network. On these bases, double dual networks are introduced to convert the individual-global-max constraint of action selection into a value range constraint of the action advantage function, thereby facilitating model learning. Simulation results showcase the superior performance of the MARL_SGAT algorithm when compared to seven baseline algorithms. Specifically, the algorithm manifests in reduced average vehicle delays, decreased number of stops, and elevated travel speeds within the controlled road network. These advantages are particularly conspicuous in heterogeneous road network environments.

Uncertainty-aware Grounded Action Transformation towards Sim-to-Real Transfer for Traffic Signal Control

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

Learning in practice: reinforcement learning-based traffic signal control augmented with actuated control

Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learning

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

Targeted Knowledge Transfer for Learning Traffic Signal Plans.

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

Safe and Psychologically Pleasant Traffic Signal Control with Reinforcement Learning using Action Masking

Distributed agent-based deep reinforcement learning for large scale traffic signal control

RA-TSC: Learning Adaptive Traffic Signal Control Strategy Via Deep Reinforcement Learning

Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach

Reference RL: Reinforcement Learning with Reference Mechanism and Its Application in Traffic Signal Control

Multi-Agent Spatial-Temporal Transformer for Traffic Signal Control

Adaptive Traffic Signal Control for Large-Scale Scenario with Cooperative Group-based Multi-agent Reinforcement Learning

First steps towards real-world traffic signal control optimisation by reinforcement learning

Reinforcement Learning Approaches for Traffic Signal Control under Missing Data

Scalable Traffic Signal Controls using Fog-Cloud Based Multiagent Reinforcement Learning

iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement

Extensible Hierarchical Multi-Agent Reinforcement-Learning Algorithm in Traffic Signal Control