Abstract:Smart traffic lights in intelligent transportation systems (ITSs) are envisioned to greatly increase traffic efficiency and reduce congestion. Deep reinforcement learning (DRL) is a promising approach to adaptively control traffic lights based on the real-time traffic situation in a road network. However, conventional methods may suffer from poor scalability. In this paper, we investigate deep reinforcement learning to control traffic lights, and both theoretical analysis and numerical experiments show that the intelligent behavior ``greenwave" (i.e., a vehicle will see a progressive cascade of green lights, and not have to brake at any intersection) emerges naturally a grid road network, which is proved to be the optimal policy in an avenue with multiple cross streets. As a first step, we use two DRL algorithms for the traffic light control problems in two scenarios. In a single road intersection, we verify that the deep Q-network (DQN) algorithm delivers a thresholding policy; and in a grid road network, we adopt the deep deterministic policy gradient (DDPG) algorithm. Secondly, numerical experiments show that the DQN algorithm delivers the optimal control, and the DDPG algorithm with passive observations has the capability to produce on its own a high-level intelligent behavior in a grid road network, namely, the ``greenwave" policy emerges. We also verify the ``greenwave" patterns in a $5 \times 10$ grid road network. Thirdly, the ``greenwave" patterns demonstrate that DRL algorithms produce favorable solutions since the ``greenwave" policy shown in experiment results is proved to be optimal in a specified traffic model (an avenue with multiple cross streets). The delivered policies both in a single road intersection and a grid road network demonstrate the scalability of DRL algorithms.

Deep Reinforcement Learning for Adaptive Traffic Signal Control

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

A Deep Reinforcement Learning Approach for Fair Traffic Signal Control

Deep reinforcement learning for traffic signal control with consistent state and reward design approach

Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

Imitation Learning Based Deep Reinforcement Learning for Traffic Signal Control

Adaptive Traffic Signal Control with Deep Reinforcement Learning An Exploratory Investigation

Deep Reinforcement Learning for Traffic Signal Control Model and Adaptation Study

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

A multi‐agent deep reinforcement learning approach for traffic signal coordination

Adaptive Traffic Signal Control With Deep Reinforcement Learning and High Dimensional Sensory Inputs: Case Study and Comprehensive Sensitivity Analyses

Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems

A survey on deep reinforcement learning approaches for traffic signal control

Optimizing Traffic Signal Control in Mixed Traffic Scenarios: A Predictive Traffic Information-Based Deep Reinforcement Learning Approach

Traffic Signal Control Using Hybrid Action Space Deep Reinforcement Learning

A Comprehensive Analysis of Reward Function for Adaptive Traffic Signal Control

A Comparison of Deep Reinforcement Learning Models for Isolated Traffic Signal Control