Abstract:In recent years, the advancement of artificial intelligence techniques has led to significant interest in reinforcement learning (RL) within the traffic and transportation community. Dynamic traffic control has emerged as a prominent application field for RL in traffic systems. This paper presents a comprehensive survey of RL studies in dynamic traffic control, addressing the challenges associated with implementing RL-based traffic control strategies in practice, and identifying promising directions for future research. The first part of this paper provides a comprehensive overview of existing studies on RL-based traffic control strategies, encompassing their model designs, training algorithms, and evaluation methods. It is found that only a few studies have isolated the training and testing environments while evaluating their RL controllers. Subsequently, we examine the challenges involved in implementing existing RL-based traffic control strategies. We investigate the learning costs associated with online RL methods and the transferability of offline RL methods through simulation experiments. The simulation results reveal that online training methods with random exploration suffer from high exploration and learning costs. Additionally, the performance of offline RL methods is highly reliant on the accuracy of the training simulator. These limitations hinder the practical implementation of existing RL-based traffic control strategies. The final part of this paper summarizes and discusses a few existing efforts which attempt to overcome these challenges. This review highlights a rising volume of studies dedicated to mitigating the limitations of RL strategies, with the specific aim of enhancing their practical implementation in recent years.

First steps towards real-world traffic signal control optimisation by reinforcement learning

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

Deep Reinforcement Learning for Adaptive Traffic Signal Control

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Reinforcement Learning Approaches for Traffic Signal Control under Missing Data

Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections

Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implementation

Deep reinforcement learning for traffic signal control with consistent state and reward design approach

Real-Time Adaptive Traffic Signal Control in a Connected and Automated Vehicle Environment: Optimisation of Signal Planning with Reinforcement Learning under Vehicle Speed Guidance

Offline Reinforcement Learning for Road Traffic Control

Learning Phase Competition for Traffic Signal Control

Learning in practice: reinforcement learning-based traffic signal control augmented with actuated control

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

Diagnosing Reinforcement Learning for Traffic Signal Control

Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion

Reinforcement Learning for Traffic Signal Control in Hybrid Action Space