Abstract:The ever-changing traffic dynamics make the traditional traffic signal control methods unable to adapt to the environment. Meanwhile, deep reinforcement learning (DRL) has the property of interacting with the environment and adapting to changes in the environment. Therefore, in recent years, researchers have usually solved traffic signal control (TSC) problems through DRL methods. They have not only improved the design of neural networks, but also improved the ability of models to understand traffic conditions and learn corresponding task requests by designing different states and rewards. However, although the existing TSC algorithms based on DRL have proposed many well-designed states and reward strategies, which combinations of states and rewards should be adopted in practice to achieve the performance margin of models remains a question that researchers are seeking the answer to. Therefore, we introduce a general simulation platform to test and compare experimental performance under different combinations of states and rewards. Specifically, we test and analyze the experimental effects under different combinations of multiple traffic states and rewards through various TSC methods with a set of unified model settings. We further design and test some new state representations and reward strategies based on more detailed traffic information. The test results show that when researchers design the state and reward, refining the traffic state like vehicle running condition and making the state and reward match can make the experimental performance better than other combinations in most cases. We hope these results have some implications for the state and reward choice when researchers conduct experiments on TSC problem or other traffic decision management problems.

A Cold-Start-free Reinforcement Learning Approach for Traffic Signal Control

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

DNLight: Learning Efficient Evaluation for Traffic Signal Control

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

Guidelines for Parameter Selection in Traffic Light Control Methods Using Reinforcement Learning: Insights from Empirical Studies

Learning in practice: reinforcement learning-based traffic signal control augmented with actuated control

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

Deep reinforcement learning for traffic signal control with consistent state and reward design approach

Deep Reinforcement Learning for Traffic Signal Control Model and Adaptation Study

Targeted Knowledge Transfer for Learning Traffic Signal Plans.

Training Reinforcement Learning Agent for Traffic Signal Control under Different Traffic Conditions

Improved Deep Reinforcement Learning for Intelligent Traffic Signal Control Using ECA_LSTM Network

Deep Reinforcement Learning for Adaptive Traffic Signal Control

A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections

A survey on deep reinforcement learning approaches for traffic signal control

Random Ensemble Reinforcement Learning for Traffic Signal Control