Abstract:Signal control has been effective to alleviate urban traffic congestion. Massive related works about signal timing optimization have been proposed and led to many signal control methods and systems. In recent years, reinforcement learning (RL) algorithms have attracted the increasing attention of researchers in the area of signal control optimization, since they can learn the optimal timing policy themselves by analyzing changing patterns between the traffic condition and signal timing plans. Multi-intersection traffic signal control problems face more challenges than classical single intersection problems such as the dimensionality issue as the joint action space of multiple agents grows exponentially with the number of intersections. The existing state-of-art multi-agent reinforcement learning algorithms developed for other areas may not suit well with traffic signal control given the complex spatial–temporal nature of traffic flows. In this paper, we propose a novel multi-agent counterfactual actor–critic with scheduler (MACS) framework for multiple interactions. In this method, decentralized actors control the traffic signals and the centralized critic combines recurrent policies with feed-forward critics. Additionally, a scheduler module that exchanges information among agents helps the individual agent better understand the entire environment. The proposed method was evaluated using simulation experiments based on a real-world urban street network in Shenzhen, China. Results showed that the proposed method outperforms the classic model-based method and several existing RL-based methods according to different measures such as queue length, average travel time delay, and throughput.

Coordinated Control of Area Traffic Signal Based on Agent

Regional Signal Coordinated Control System Based on an Ant Algorithm

Adaptive signal control for urban traffic network gridlock

A Multi-Agent Coordinate Model for Urban Traffic Signal Control

An Approach to Urban Road Traffic Signal Coordination Control Based on Multi-Agent System

Multi-agent Fuzzy Signal Control with Bus Priority

Study of intersection traffic control based on agent

Coordination based on Multi-Agent and its application of urban intersection traffic coordination control

An Intelligent Traffic Control Model Based on Intersection Agent

Dynamic intelligent coordinated control with arterial traffic based on multi-agent

An Approach to Urban Traffic Network Signal Optimization Based Control on Multi-agent Coordination

Traffic trunk roads coordination control based on multi-agent

Urban Traffic Area Control Based on Muti-Agent System

New Methods for Research in Multi-Agent Traffic Signal Control

Urban Traffic Coordination Control System Based on Multi-Agent

Optimizing multi-agent based urban traffic signal control system

A Game Theoretical Model of Multi-Agents in Area Coordination and Optimization of Traffic Signals

A New Traffic Signal Coordinated Control Method Based on Mobile Agent Technology

Multi-agent Coordination Algorithm for Traffic Control

The coordination between traffic signal control agents based on Q-learning

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach