Abstract:With the increasing availability of traffic data and advance of deep reinforcement learning techniques, there is an emerging trend of employing reinforcement learning (RL) for traffic signal control. A key question for applying RL to traffic signal control is how to define the reward and state. The ultimate objective in traffic signal control is to minimize the travel time, which is difficult to reach directly. Hence, existing studies often define reward as an ad-hoc weighted linear combination of several traffic measures. However, there is no guarantee that the travel time will be optimized with the reward. In addition, recent RL approaches use more complicated state (e.g., image) in order to describe the full traffic situation. However, none of the existing studies has discussed whether such a complex state representation is necessary. This extra complexity may lead to significantly slower learning process but may not necessarily bring significant performance gain. In this paper, we propose to re-examine the RL approaches through the lens of classic transportation theory. We ask the following questions: (1) How should we design the reward so that one can guarantee to minimize the travel time? (2) How to design a state representation which is concise yet sufficient to obtain the optimal solution? Our proposed method LIT is theoretically supported by the classic traffic signal control methods in transportation field. LIT has a very simple state and reward design, thus can serve as a building block for future RL approaches to traffic signal control. Extensive experiments on both synthetic and real datasets show that our method significantly outperforms the state-of-the-art traffic signal control methods.

Learning Traffic Signal Control from Demonstrations

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Hierarchically and Cooperatively Learning Traffic Signal Control

Traffic Signal Timing via Parallel Reinforcement Learning

Imitation Learning Based Deep Reinforcement Learning for Traffic Signal Control

Diagnosing Reinforcement Learning for Traffic Signal Control

Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control

Reinforcement Learning Approaches for Traffic Signal Control under Missing Data

ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control

DataLight: Offline Data-Driven Traffic Signal Control

A Comparison of Deep Reinforcement Learning Models for Isolated Traffic Signal Control

A Method for High-Value Driving Demonstration Data Generation Based on One-Dimensional Deep Convolutional Generative Adversarial Networks

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

A Novel Multi-Agent Deep RL Approach for Traffic Signal Control

ADLight: A Universal Approach of Traffic Signal Control with Augmented Data Using Reinforcement Learning

GuideLight: "Industrial Solution" Guidance for More Practical Traffic Signal Control Agents

DERLight: A Deep Reinforcement Learning Traffic Light Control Algorithm with Dual Experience Replay

MTLight: Efficient Multi-Task Reinforcement Learning for Traffic Signal Control