Abstract:Smart traffic lights in intelligent transportation systems (ITSs) are envisioned to greatly increase traffic efficiency and reduce congestion. Deep reinforcement learning (DRL) is a promising approach to adaptively control traffic lights based on the real-time traffic situation in a road network. However, conventional methods may suffer from poor scalability. In this paper, we investigate deep reinforcement learning to control traffic lights, and both theoretical analysis and numerical experiments show that the intelligent behavior ``greenwave" (i.e., a vehicle will see a progressive cascade of green lights, and not have to brake at any intersection) emerges naturally a grid road network, which is proved to be the optimal policy in an avenue with multiple cross streets. As a first step, we use two DRL algorithms for the traffic light control problems in two scenarios. In a single road intersection, we verify that the deep Q-network (DQN) algorithm delivers a thresholding policy; and in a grid road network, we adopt the deep deterministic policy gradient (DDPG) algorithm. Secondly, numerical experiments show that the DQN algorithm delivers the optimal control, and the DDPG algorithm with passive observations has the capability to produce on its own a high-level intelligent behavior in a grid road network, namely, the ``greenwave" policy emerges. We also verify the ``greenwave" patterns in a $5 \times 10$ grid road network. Thirdly, the ``greenwave" patterns demonstrate that DRL algorithms produce favorable solutions since the ``greenwave" policy shown in experiment results is proved to be optimal in a specified traffic model (an avenue with multiple cross streets). The delivered policies both in a single road intersection and a grid road network demonstrate the scalability of DRL algorithms.

Leveraging Deep Reinforcement Learning for Traffic Engineering: A Survey

Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implementation

Enabling efficient routing for traffic engineering in SDN with Deep Reinforcement Learning

Experience-driven Networking: A Deep Reinforcement Learning based Approach

A survey on deep reinforcement learning approaches for traffic signal control

Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey

TITE: A Transformer-Based Deep Reinforcement Learning Approach for Traffic Engineering in Hybrid SDN with Dynamic Traffic

Deep Reinforcement Learning-Based Traffic Engineering in Cloud-Edge-End Collaboration Environments

Distributed and Adaptive Traffic Engineering with Deep Reinforcement Learning

Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Freeway network traffic management based on distributed reinforcement learning

An SDN Traffic Engineering Approach Based on Traffic Unsupervised Contrastive Representation and Reinforcement Learning

Deep Reinforcement Learning for Adaptive Traffic Signal Control

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

Dynamically Split the Traffic in Software Defined Network Based on Deep Reinforcement Learning.

Deep Reinforcement Learning and Transportation Research: A Comprehensive Review

A Survey on Deep Learning-Based Traffic Signal Control

Recent Advances in Reinforcement Learning for Traffic Signal Control