Abstract:Traffic Signal Control (TSC) plays a crucial role in mitigating congestion. The extensive integration of Deep Reinforcement Learning (DRL) into TSC, fueled by the advancements in artificial intelligence, has yielded remarkable efficacy. Nevertheless, prevailing DRL models are typically developed within homogeneous road networks, characterized by identical intersection types and relatively balanced traffic flow patterns. This predominant approach often results in an oversight of the impact of intersection heterogeneity, encompassing variations in intersection geometric structure, traffic demands, signal timing design, and other pertinent factors, on action selections by researchers. This paper addresses the aforementioned gap by introducing a value-decomposition-based spatiotemporal graph attention multi-agent DRL model (MARL_SGAT) that expressly accommodates intersection heterogeneity. Specifically, a heterogeneous correlation index is designed using road network structural parameters and serves to formulate a heterogeneous reward function, enabling the quantification of action execution improvements in diverse road network environments. To address the intricate spatiotemporal features of traffic flows within the heterogeneous road network, the model strategically selects and incorporates these features during the formulation by leveraging a spatiotemporal graph attention network. On these bases, double dual networks are introduced to convert the individual-global-max constraint of action selection into a value range constraint of the action advantage function, thereby facilitating model learning. Simulation results showcase the superior performance of the MARL_SGAT algorithm when compared to seven baseline algorithms. Specifically, the algorithm manifests in reduced average vehicle delays, decreased number of stops, and elevated travel speeds within the controlled road network. These advantages are particularly conspicuous in heterogeneous road network environments.

CoSLight: Co-optimizing Collaborator Selection and Decision-making to Enhance Traffic Signal Control

CoLight: Learning Network-level Cooperation for Traffic Signal Control

CLlight: Enhancing representation of multi-agent reinforcement learning with contrastive learning for cooperative traffic signal control

CityLight: A Universal Model for Coordinated Traffic Signal Control in City-scale Heterogeneous Intersections

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Cooperative Reinforcement Learning on Traffic Signal Control

Hierarchically and Cooperatively Learning Traffic Signal Control

FedLight: Federated Reinforcement Learning for Autonomous Multi-Intersection Traffic Signal Control

LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environments

Cooperative Control for Multi-Intersection Traffic Signal Based on Deep Reinforcement Learning and Imitation Learning

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

Two-layer Coordinated Reinforcement Learning for Traffic Signal Control in Traffic Network

SafeLight: A Reinforcement Learning Method toward Collision-free Traffic Signal Control

Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control

MTLight: Efficient Multi-Task Reinforcement Learning for Traffic Signal Control

CycLight: learning traffic signal cooperation with a cycle-level strategy

MixLight: Mixed-Agent Cooperative Reinforcement Learning for Traffic Light Control

Cooperative Traffic Signal Control Using a Distributed Agent-Based Deep Reinforcement Learning With Incentive Communication

MalLight: Influence-Aware Coordinated Traffic Signal Control for Traffic Signal Malfunctions

Integrated decision and control at multi-lane intersections with mixed traffic flow

Graph cooperation deep reinforcement learning for ecological urban traffic signal control