Abstract:Traffic Signal Control (TSC) plays a crucial role in mitigating congestion. The extensive integration of Deep Reinforcement Learning (DRL) into TSC, fueled by the advancements in artificial intelligence, has yielded remarkable efficacy. Nevertheless, prevailing DRL models are typically developed within homogeneous road networks, characterized by identical intersection types and relatively balanced traffic flow patterns. This predominant approach often results in an oversight of the impact of intersection heterogeneity, encompassing variations in intersection geometric structure, traffic demands, signal timing design, and other pertinent factors, on action selections by researchers. This paper addresses the aforementioned gap by introducing a value-decomposition-based spatiotemporal graph attention multi-agent DRL model (MARL_SGAT) that expressly accommodates intersection heterogeneity. Specifically, a heterogeneous correlation index is designed using road network structural parameters and serves to formulate a heterogeneous reward function, enabling the quantification of action execution improvements in diverse road network environments. To address the intricate spatiotemporal features of traffic flows within the heterogeneous road network, the model strategically selects and incorporates these features during the formulation by leveraging a spatiotemporal graph attention network. On these bases, double dual networks are introduced to convert the individual-global-max constraint of action selection into a value range constraint of the action advantage function, thereby facilitating model learning. Simulation results showcase the superior performance of the MARL_SGAT algorithm when compared to seven baseline algorithms. Specifically, the algorithm manifests in reduced average vehicle delays, decreased number of stops, and elevated travel speeds within the controlled road network. These advantages are particularly conspicuous in heterogeneous road network environments.

The Study of Reinforcement Learning for Traffic Self-Adaptive Control under Multiagent Markov Game Environment

Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

Urban Traffic Light Control Via Active Multi-agent Communication and Supply-Demand Modeling

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

Traffic Signal Timing via Parallel Reinforcement Learning

A multi-agent reinforcement learning method with curriculum transfer for large-scale dynamic traffic signal control

The coordination between traffic signal control agents based on Q-learning

Multi-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach

Mean Field Multi-Agent Reinforcement Learning Method for Area Traffic Signal Control

A Cooperative Multiagent System for Traffic Signal Control Using Game Theory and Reinforcement Learning

Distributed agent-based deep reinforcement learning for large scale traffic signal control

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

An Information Fusion Approach to Intelligent Traffic Signal Control Using the Joint Methods of Multiagent Reinforcement Learning and Artificial Intelligence of Things

Batch-Augmented Multi-Agent Reinforcement Learning for Efficient Traffic Signal Optimization

A multi‐agent deep reinforcement learning approach for traffic signal coordination

Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

Deep Reinforcement Learning for Traffic Signal Control along Arterials

Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network

Adaptive Traffic Signal Control for Large-Scale Scenario with Cooperative Group-based Multi-agent Reinforcement Learning

Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning