Abstract:The increasingly severe congestion problem in modern cities strengthens the significance of developing city-scale traffic signal control (TSC) methods for traffic efficiency enhancement. While reinforcement learning has been widely explored in TSC, most of them still target small-scale optimization and cannot directly scale to the city level due to unbearable resource demand. Only a few of them manage to tackle city-level optimization, namely a thousand-scale optimization, by incorporating parameter-sharing mechanisms, but hardly have they fully tackled the heterogeneity of intersections and intricate between-intersection interactions inherent in real-world city road networks. To fill in the gap, we target at the two important challenges in adopting parameter-sharing paradigms to solve TSC: inconsistency of inner state representations for intersections heterogeneous in configuration, scale, and orders of available traffic phases; intricacy of impacts from neighborhood intersections that have various relative traffic relationships due to inconsistent phase orders and diverse relative positioning. Our method, CityLight, features a universal representation module that not only aligns the state representations of intersections by reindexing their phases based on their semantics and designing heterogeneity-preserving observations, but also encodes the narrowed relative traffic relation types to project the neighborhood intersections onto a uniform relative traffic impact space. We further attentively fuse neighborhood representations based on their competing relations and incorporate neighborhood-integrated rewards to boost coordination. Extensive experiments with hundreds to tens of thousands of intersections validate the surprising effectiveness and generalizability of CityLight, with an overall performance gain of 11.68% and a 22.59% improvement in transfer scenarios in throughput.

PhaseLight: an Universal and Practical Traffic Signal Control Algorithms Based on Reinforcement Learning

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

DNLight: Learning Efficient Evaluation for Traffic Signal Control

DynamicLight: Two-Stage Dynamic Traffic Signal Timing

DynamicLight: Dynamically Tuning Traffic Signal Duration with DRL

Learning Phase Competition for Traffic Signal Control

CycLight: learning traffic signal cooperation with a cycle-level strategy

ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control

Learning in practice: reinforcement learning-based traffic signal control augmented with actuated control

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

ADLight: A Universal Approach of Traffic Signal Control with Augmented Data Using Reinforcement Learning

MoveLight: Enhancing Traffic Signal Control through Movement-Centric Deep Reinforcement Learning

CityLight: A Universal Model for Coordinated Traffic Signal Control in City-scale Heterogeneous Intersections

Hierarchically and Cooperatively Learning Traffic Signal Control

GuideLight: "Industrial Solution" Guidance for More Practical Traffic Signal Control Agents

Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control

SafeLight: A Reinforcement Learning Method toward Collision-free Traffic Signal Control

UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control

Π-Light: Programmatic Interpretable Reinforcement Learning for Resource-Limited Traffic Signal Control

DataLight: Offline Data-Driven Traffic Signal Control

A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections