Abstract:Cooperation between intersections in large-scale road networks is critical in traffic congestion. Currently, most traffic signals cooperate via pre-defined timing phases, which is extremely inefficient in real-time traffic scenarios. Most existing studies on multi-agent reinforcement learning (MARL) traffic signal control have focused on designing efficient communication methods, but have ignored the importance of how agents interact in cooperative communication. To achieve more efficient cooperation among traffic signals and alleviate urban traffic congestion, this study constructs a Graph Cooperation Q-learning Network Traffic Signal Control (GCQN-TSC) model, which is a graph cooperation network with an embedded self-attention mechanism that enables agents to adjust their attention in real time according to the dynamic traffic flow information, perceive the traffic environment quickly and effectively in a larger range, and help agents achieve more effective collaboration. Moreover, the Deep Graph Q-learning (DGQ) algorithm is proposed in this model to optimize the traffic signal control strategy according to the spatio-temporal characteristics of different traffic scenes and provide the optimal signal phase for each intersection. This study also integrates the ecological traffic concept into MARL traffic signal control, which aims to reduce traffic exhaust emissions. Finally, the proposed GCQN-TSC is experimentally validated both in a synthetic traffic grid and a real-world traffic network using the SUMO simulator. The experimental results show that GCQN-TSC outperforms other traffic signal control methods in almost all performance metrics, including average queue length and waiting time, as it can aggregate information acquired from collaborative agents and make network-level signal optimization decisions.

The coordination between traffic signal control agents based on Q-learning

TraCo: Learning Virtual Traffic Coordinator for Cooperation with Multi-Agent Reinforcement Learning.

A multi‐agent deep reinforcement learning approach for traffic signal coordination

The Study of Reinforcement Learning for Traffic Self-Adaptive Control under Multiagent Markov Game Environment

An Approach to Urban Road Traffic Signal Coordination Control Based on Multi-Agent System

Coordination based on Multi-Agent and its application of urban intersection traffic coordination control

Traffic Signal Timing via Parallel Reinforcement Learning

Urban Road Traffic Signal Control Method Based on Agent

Method of Intersection Agent Coordination Based on Game-learning

Intelligent Traffic Light Control Using Distributed Multi-agent Q Learning

Traffic Signal Control Agent Interaction Model Based on Game Theory and Reinforcement Learning

Large-Scale Traffic Signal Control Using a Novel Multiagent Reinforcement Learning

A Method of Reinforcement Learning Based Automatic Traffic Signal Control

Dynamic Q-Learning for Intersection Traffic Flow Control Based on Agents

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

Multi-Agent urban traffic coordination control research based on game learning

Graph cooperation deep reinforcement learning for ecological urban traffic signal control

Multi-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach

Coordination of Urban Intersection Agents Based on Multi-interaction History Learning Method

Learning scalable multi-agent coordination by spatial differentiation for traffic signal control

Cooperative Traffic Signal Control Using a Distributed Agent-Based Deep Reinforcement Learning With Incentive Communication