Abstract:The ever-changing traffic dynamics make the traditional traffic signal control methods unable to adapt to the environment. Meanwhile, deep reinforcement learning (DRL) has the property of interacting with the environment and adapting to changes in the environment. Therefore, in recent years, researchers have usually solved traffic signal control (TSC) problems through DRL methods. They have not only improved the design of neural networks, but also improved the ability of models to understand traffic conditions and learn corresponding task requests by designing different states and rewards. However, although the existing TSC algorithms based on DRL have proposed many well-designed states and reward strategies, which combinations of states and rewards should be adopted in practice to achieve the performance margin of models remains a question that researchers are seeking the answer to. Therefore, we introduce a general simulation platform to test and compare experimental performance under different combinations of states and rewards. Specifically, we test and analyze the experimental effects under different combinations of multiple traffic states and rewards through various TSC methods with a set of unified model settings. We further design and test some new state representations and reward strategies based on more detailed traffic information. The test results show that when researchers design the state and reward, refining the traffic state like vehicle running condition and making the state and reward match can make the experimental performance better than other combinations in most cases. We hope these results have some implications for the state and reward choice when researchers conduct experiments on TSC problem or other traffic decision management problems.

A Survey on Reinforcement Learning Models and Algorithms for Traffic Signal Control

Recent Advances in Reinforcement Learning for Traffic Signal Control

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections

Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implementation

Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion

A survey on deep reinforcement learning approaches for traffic signal control

A survey on reinforcement learning-based control for signalized intersections with connected automated vehicles

Reinforcement Learning Approaches for Traffic Signal Control under Missing Data

A Survey on Traffic Signal Control Methods.

Reinforcement Learning in Urban Network Traffic Signal Control: A Systematic Literature Review

Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

Diagnosing Reinforcement Learning for Traffic Signal Control

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

A Traffic Signal Control Method Based on Asynchronous Reinforcement Learning

Guidelines for Parameter Selection in Traffic Light Control Methods Using Reinforcement Learning: Insights from Empirical Studies

Deep Reinforcement Learning for Adaptive Traffic Signal Control

A Comparison of Deep Reinforcement Learning Models for Isolated Traffic Signal Control

Learning Phase Competition for Traffic Signal Control

Effects analysis of reward functions on reinforcement learning for traffic signal control