Abstract:The recent advancements in Traffic Signal Control (TSC) have highlighted the potential of Reinforcement Learning (RL) as a promising solution to alleviate traffic congestion. Current research in this area primarily concentrates on either online or offline learning strategies, aiming to create optimized policies for specific cities. Nevertheless, the transferability of these policies to new cities is impeded by constraints such as the limited availability of high-quality data and the expensive and risky exploration process. To this end, in this paper, we present an innovative cross-city Traffic Signal Control (TSC) paradigm called CrossLight. Our approach involves meta training using offline data from source cities and adaptively fine-tuning in the target city. This novel methodology aims to address the challenges of transferring TSC policies across different cities effectively. In our proposed approach, we start by acquiring meta-decision pattern knowledge through trajectory dynamics reconstruction via pre-training in source cities. To address disparities in road network topologies between cities, we dynamically construct city topological structures based on the extracted meta-knowledge during the offline meta-training phase. These structures are then used to distill pattern-structure aware representations of decision trajectories from the source cities. To identify effective initial parameters for the learnable components, we employ the Model-Agnostic Meta-Learning (MAML) framework, a popular meta-learning approach. During adaptive fine-tuning in the target city, we introduce a replay buffer that is iteratively updated using online interactions with a rank and filter mechanism. This mechanism, along with a carefully designed exploration strategy, ensures a balance between exploitation and exploration, thereby fostering both the diversity and quality of the trajectories for fine-tuning. Finally, extensive experiments across four cities validate that CrossLight achieves comparable performance in new cities with minimal fine-tuning iterations, surpassing both existing online and offline methods. This success underscores that our CrossLight framework emerges as a groundbreaking and potent paradigm, offering a feasible and effective solution to the intelligent transportation community.

Brief Industry Paper: RTLight: Digital Twin-Based Real-Time Federated Traffic Signal Control

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

DNLight: Learning Efficient Evaluation for Traffic Signal Control

FedLight: Federated Reinforcement Learning for Autonomous Multi-Intersection Traffic Signal Control

A scalable approach to optimize traffic signal control with federated reinforcement learning

DataLight: Offline Data-Driven Traffic Signal Control

PhaseLight: an Universal and Practical Traffic Signal Control Algorithms Based on Reinforcement Learning

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

CrossLight: Offline-to-Online Reinforcement Learning for Cross-City Traffic Signal Control

Learning in practice: reinforcement learning-based traffic signal control augmented with actuated control

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control

DynamicLight: Two-Stage Dynamic Traffic Signal Timing

UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control

DynamicLight: Dynamically Tuning Traffic Signal Duration with DRL

IPDALight: Intensity- and Phase Duration-Aware Traffic Signal Control Based on Reinforcement Learning

A survey on deep reinforcement learning approaches for traffic signal control

MTLight: Efficient Multi-Task Reinforcement Learning for Traffic Signal Control

Hierarchically and Cooperatively Learning Traffic Signal Control

ADLight: A Universal Approach of Traffic Signal Control with Augmented Data Using Reinforcement Learning