Abstract:Cooperation between intersections in large-scale road networks is critical in traffic congestion. Currently, most traffic signals cooperate via pre-defined timing phases, which is extremely inefficient in real-time traffic scenarios. Most existing studies on multi-agent reinforcement learning (MARL) traffic signal control have focused on designing efficient communication methods, but have ignored the importance of how agents interact in cooperative communication. To achieve more efficient cooperation among traffic signals and alleviate urban traffic congestion, this study constructs a Graph Cooperation Q-learning Network Traffic Signal Control (GCQN-TSC) model, which is a graph cooperation network with an embedded self-attention mechanism that enables agents to adjust their attention in real time according to the dynamic traffic flow information, perceive the traffic environment quickly and effectively in a larger range, and help agents achieve more effective collaboration. Moreover, the Deep Graph Q-learning (DGQ) algorithm is proposed in this model to optimize the traffic signal control strategy according to the spatio-temporal characteristics of different traffic scenes and provide the optimal signal phase for each intersection. This study also integrates the ecological traffic concept into MARL traffic signal control, which aims to reduce traffic exhaust emissions. Finally, the proposed GCQN-TSC is experimentally validated both in a synthetic traffic grid and a real-world traffic network using the SUMO simulator. The experimental results show that GCQN-TSC outperforms other traffic signal control methods in almost all performance metrics, including average queue length and waiting time, as it can aggregate information acquired from collaborative agents and make network-level signal optimization decisions.

Multi-Agent urban traffic coordination control research based on game learning

Adaptive signal control for urban traffic network gridlock

TraCo: Learning Virtual Traffic Coordinator for Cooperation with Multi-Agent Reinforcement Learning.

Adaptive algorithm for multi-agent learning optimal cooperative pursuit strategy based on Markov game

The coordination between traffic signal control agents based on Q-learning

Learning scalable multi-agent coordination by spatial differentiation for traffic signal control

Urban Traffic Light Control Via Active Multi-Agent Communication and Supply-Demand Modeling

A Cooperative Multiagent System for Traffic Signal Control Using Game Theory and Reinforcement Learning

Cooperative Path Following Control in Autonomous Vehicles Graphical Games: A Data-Based Off-Policy Learning Approach

A Multi-Agent Coordinate Model for Urban Traffic Signal Control

Autonomous driving planning and decision making based on game theory and reinforcement learning

Simulation research on urban traffic integration control system based on multi-agent

Beyond centralization: Non-cooperative perimeter control with extended mean-field reinforcement learning in urban road networks

Combat Urban Congestion via Collaboration: Heterogeneous GNN-based MARL for Coordinated Platooning and Traffic Signal Control

Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning

Graph cooperation deep reinforcement learning for ecological urban traffic signal control

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

Two-layer Coordinated Reinforcement Learning for Traffic Signal Control in Traffic Network

A multi‐agent deep reinforcement learning approach for traffic signal coordination

Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control

Multi-Agent Attention Double Actor-Critic Framework for Intelligent Traffic Light Control in Urban Scenarios With Hybrid Traffic