Abstract:Traffic signal control systems (TSCSs) are integral to intelligent traffic management, fostering efficient vehicle flow. Traditional approaches often simplify road networks into standard graphs, which results in a failure to consider the dynamic nature of traffic data at neighboring intersections, thereby neglecting higher-order interconnections necessary for real-time control. To address this, we propose a novel TSCS framework to realize intelligent traffic control. This framework collaborates with multiple neighboring edge computing servers to collect traffic information across the road network. To elevate the efficiency of traffic signal control, we have crafted a multi-agent soft actor-critic (MA-SAC) reinforcement learning algorithm. Within this algorithm, individual agents are deployed at each intersection with a mandate to optimize traffic flow across the entire road network collectively. Furthermore, we introduce hypergraph learning into the critic network of MA-SAC to enable the spatio-temporal interactions from multiple intersections in the road network. This method fuses hypergraph and spatio-temporal graph structures to encode traffic data and capture the complex spatial and temporal correlations between multiple intersections. Our empirical evaluation, tested on varied datasets, demonstrates the superiority of our framework in minimizing average vehicle travel times and sustaining high-throughput performance. This work facilitates the development of more intelligent and reactive urban traffic management solutions.

What problem does this paper attempt to address?

The paper aims to address the issues of Traffic Signal Control Systems (TSCSs) in real-time control and optimization of urban traffic flow. Traditional TSCSs simplify the road network structure, treating it as a standard graph, but this simplification overlooks the higher-order interconnectivity of dynamic traffic data between adjacent intersections, leading to an inability to perform real-time control. To tackle this issue, the paper proposes a novel TSCS framework based on the Multi-Agent Soft Actor-Critic (MA-SAC) reinforcement learning algorithm. This framework collaborates with multiple edge computing servers to collect traffic information on the road network and utilizes the MA-SAC algorithm to optimize the traffic flow at each intersection, thereby achieving collective optimization of the entire network. Furthermore, the paper introduces hypergraph learning into the critic network of MA-SAC to capture the spatial and temporal interactions among multiple intersections in the road network. This approach combines hypergraph and spatio-temporal graph structures, encoding traffic data and capturing the complex spatial and temporal correlations between multiple intersections. Experimental evaluations show that the framework outperforms existing methods in reducing average vehicle travel time and maintaining high throughput performance, promoting the development of smarter and more responsive urban traffic management solutions. Specifically, the contributions of the paper include: 1. Proposing an edge intelligence-based framework that divides the entire road network into multiple regions and deploys a Multi-access Edge Computing (MEC) server in each region for facilitating traffic information exchange and training of multiple agents. 2. Adopting the Multi-Agent Soft Actor-Critic (MA-SAC) reinforcement learning algorithm, coordinating traffic condition control at multiple intersections by deploying an agent at each intersection. 3. Introducing a hypergraph learning module that dynamically constructs spatio-temporal hyperedges to capture the spatio-temporal dependencies among multiple traffic signals, marking the first introduction of the hypergraph concept in the intelligent TSCS domain. 4. Conducting extensive experiments on synthetic and real-world traffic datasets, with results proving that the proposed solution surpasses existing technologies on various metrics such as average travel time and throughput. In summary, the paper proposes an innovative framework that effectively addresses urban traffic challenges by combining multi-agent reinforcement learning, hypergraph learning, and edge computing.

Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs

Multi-agent Deep Reinforcement Learning with Spatio-Temporal Feature Fusion for Traffic Signal Control

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity

The Study of Reinforcement Learning for Traffic Self-Adaptive Control under Multiagent Markov Game Environment

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

Heterogeneous Multi-Agent Reinforcement Learning Based on Adaptive Curiosity for Traffic Signal Control

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning

Graph cooperation deep reinforcement learning for ecological urban traffic signal control

Multi-Agent Spatial-Temporal Transformer for Traffic Signal Control

Distributed agent-based deep reinforcement learning for large scale traffic signal control

Multi-Objective Traffic Signal Control Using Network-Wide Agent Coordinated Reinforcement Learning.

Graph Topography-Aware Reinforcement Learning for Intelligent Traffic Signal Control

Real-Time Network-Level Traffic Signal Control: An Explicit Multiagent Coordination Method

Scalable Traffic Signal Controls using Fog-Cloud Based Multiagent Reinforcement Learning

An integrated and cooperative architecture for multi-intersection traffic signal control

A Spatial-Temporal Graph Attention Network for Multi-intersection Traffic Light Control

Extensible Hierarchical Multi-Agent Reinforcement-Learning Algorithm in Traffic Signal Control

An Information Fusion Approach to Intelligent Traffic Signal Control Using the Joint Methods of Multiagent Reinforcement Learning and Artificial Intelligence of Things

STMARL: A Spatio-Temporal Multi-Agent Reinforcement Learning Approach for Cooperative Traffic Light Control