Abstract:The ever-changing traffic dynamics make the traditional traffic signal control methods unable to adapt to the environment. Meanwhile, deep reinforcement learning (DRL) has the property of interacting with the environment and adapting to changes in the environment. Therefore, in recent years, researchers have usually solved traffic signal control (TSC) problems through DRL methods. They have not only improved the design of neural networks, but also improved the ability of models to understand traffic conditions and learn corresponding task requests by designing different states and rewards. However, although the existing TSC algorithms based on DRL have proposed many well-designed states and reward strategies, which combinations of states and rewards should be adopted in practice to achieve the performance margin of models remains a question that researchers are seeking the answer to. Therefore, we introduce a general simulation platform to test and compare experimental performance under different combinations of states and rewards. Specifically, we test and analyze the experimental effects under different combinations of multiple traffic states and rewards through various TSC methods with a set of unified model settings. We further design and test some new state representations and reward strategies based on more detailed traffic information. The test results show that when researchers design the state and reward, refining the traffic state like vehicle running condition and making the state and reward match can make the experimental performance better than other combinations in most cases. We hope these results have some implications for the state and reward choice when researchers conduct experiments on TSC problem or other traffic decision management problems.

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

DynamicLight: Dynamically Tuning Traffic Signal Duration with DRL

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

DynamicLight: Two-Stage Dynamic Traffic Signal Timing

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

Guidelines for Parameter Selection in Traffic Light Control Methods Using Reinforcement Learning: Insights from Empirical Studies

DataLight: Offline Data-Driven Traffic Signal Control

Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach

Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control

ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control

Deep Reinforcement Learning for Traffic Signal Control Model and Adaptation Study

SafeLight: A Reinforcement Learning Method toward Collision-free Traffic Signal Control

PhaseLight: an Universal and Practical Traffic Signal Control Algorithms Based on Reinforcement Learning

An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning

Improved Deep Reinforcement Learning for Intelligent Traffic Signal Control Using ECA_LSTM Network

SoftLight: A Maximum Entropy Deep Reinforcement Learning Approach for Intelligent Traffic Signal Control

Leveraging Queue Length and Attention Mechanisms for Enhanced Traffic Signal Control Optimization