Abstract:The recent advancements in Traffic Signal Control (TSC) have highlighted the potential of Reinforcement Learning (RL) as a promising solution to alleviate traffic congestion. Current research in this area primarily concentrates on either online or offline learning strategies, aiming to create optimized policies for specific cities. Nevertheless, the transferability of these policies to new cities is impeded by constraints such as the limited availability of high-quality data and the expensive and risky exploration process. To this end, in this paper, we present an innovative cross-city Traffic Signal Control (TSC) paradigm called CrossLight. Our approach involves meta training using offline data from source cities and adaptively fine-tuning in the target city. This novel methodology aims to address the challenges of transferring TSC policies across different cities effectively. In our proposed approach, we start by acquiring meta-decision pattern knowledge through trajectory dynamics reconstruction via pre-training in source cities. To address disparities in road network topologies between cities, we dynamically construct city topological structures based on the extracted meta-knowledge during the offline meta-training phase. These structures are then used to distill pattern-structure aware representations of decision trajectories from the source cities. To identify effective initial parameters for the learnable components, we employ the Model-Agnostic Meta-Learning (MAML) framework, a popular meta-learning approach. During adaptive fine-tuning in the target city, we introduce a replay buffer that is iteratively updated using online interactions with a rank and filter mechanism. This mechanism, along with a carefully designed exploration strategy, ensures a balance between exploitation and exploration, thereby fostering both the diversity and quality of the trajectories for fine-tuning. Finally, extensive experiments across four cities validate that CrossLight achieves comparable performance in new cities with minimal fine-tuning iterations, surpassing both existing online and offline methods. This success underscores that our CrossLight framework emerges as a groundbreaking and potent paradigm, offering a feasible and effective solution to the intelligent transportation community.

Temporality Spatialization: A Scalable and Faithful Time-Travelling Visualization for Deep Classifier Training.

TinyLight: Adaptive Traffic Signal Control on Devices with Extremely Limited Resources

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

DNLight: Learning Efficient Evaluation for Traffic Signal Control

DynamicLight: Dynamically Tuning Traffic Signal Duration with DRL

Π-Light: Programmatic Interpretable Reinforcement Learning for Resource-Limited Traffic Signal Control

Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach

DataLight: Offline Data-Driven Traffic Signal Control

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

Deep Learning-driven Mobile Traffic Measurement Collection and Analysis

DynamicLight: Two-Stage Dynamic Traffic Signal Timing

MTLight: Efficient Multi-Task Reinforcement Learning for Traffic Signal Control

CrossLight: Offline-to-Online Reinforcement Learning for Cross-City Traffic Signal Control

ADLight: A Universal Approach of Traffic Signal Control with Augmented Data Using Reinforcement Learning

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

Spatio-Temporal Sequence Modeling for Traffic Signal Control

DERLight: A Deep Reinforcement Learning Traffic Light Control Algorithm with Dual Experience Replay

TransformerLight: A Novel Sequence Modeling Based Traffic Signaling Mechanism Via Gated Transformer.

Scalable Traffic Signal Controls using Fog-Cloud Based Multiagent Reinforcement Learning

Deep Q learning-based traffic signal control algorithms: Model development and evaluation with field data

Multi-agent Deep Reinforcement Learning collaborative Traffic Signal Control method considering intersection heterogeneity