Abstract:Cooperation between intersections in large-scale road networks is critical in traffic congestion. Currently, most traffic signals cooperate via pre-defined timing phases, which is extremely inefficient in real-time traffic scenarios. Most existing studies on multi-agent reinforcement learning (MARL) traffic signal control have focused on designing efficient communication methods, but have ignored the importance of how agents interact in cooperative communication. To achieve more efficient cooperation among traffic signals and alleviate urban traffic congestion, this study constructs a Graph Cooperation Q-learning Network Traffic Signal Control (GCQN-TSC) model, which is a graph cooperation network with an embedded self-attention mechanism that enables agents to adjust their attention in real time according to the dynamic traffic flow information, perceive the traffic environment quickly and effectively in a larger range, and help agents achieve more effective collaboration. Moreover, the Deep Graph Q-learning (DGQ) algorithm is proposed in this model to optimize the traffic signal control strategy according to the spatio-temporal characteristics of different traffic scenes and provide the optimal signal phase for each intersection. This study also integrates the ecological traffic concept into MARL traffic signal control, which aims to reduce traffic exhaust emissions. Finally, the proposed GCQN-TSC is experimentally validated both in a synthetic traffic grid and a real-world traffic network using the SUMO simulator. The experimental results show that GCQN-TSC outperforms other traffic signal control methods in almost all performance metrics, including average queue length and waiting time, as it can aggregate information acquired from collaborative agents and make network-level signal optimization decisions.

A Cooperative Multiagent System for Traffic Signal Control Using Game Theory and Reinforcement Learning

Adaptive signal control for urban traffic network gridlock

A multi-agent reinforcement learning based approach for intelligent traffic signal control

Multi-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

An Information Fusion Approach to Intelligent Traffic Signal Control Using the Joint Methods of Multiagent Reinforcement Learning and Artificial Intelligence of Things

A multi‐agent deep reinforcement learning approach for traffic signal coordination

Cooperative Traffic Signal Control Using a Distributed Agent-Based Deep Reinforcement Learning With Incentive Communication

Urban Traffic Light Control Via Active Multi-agent Communication and Supply-Demand Modeling

A Multi-Agent Coordinate Model for Urban Traffic Signal Control

A multi-agent reinforcement learning method with curriculum transfer for large-scale dynamic traffic signal control

Distributed agent-based deep reinforcement learning for large scale traffic signal control

Hierarchical traffic signal optimization using reinforcement learning and traffic prediction with long-short term memory

Optimizing multi-agent based urban traffic signal control system

Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction

Distributed Signal Control of Arterial Corridors Using Multi-Agent Deep Reinforcement Learning

Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network

Holonic multi-agent system for traffic signals control

Graph cooperation deep reinforcement learning for ecological urban traffic signal control

The coordination between traffic signal control agents based on Q-learning