Abstract:The ever-changing traffic dynamics make the traditional traffic signal control methods unable to adapt to the environment. Meanwhile, deep reinforcement learning (DRL) has the property of interacting with the environment and adapting to changes in the environment. Therefore, in recent years, researchers have usually solved traffic signal control (TSC) problems through DRL methods. They have not only improved the design of neural networks, but also improved the ability of models to understand traffic conditions and learn corresponding task requests by designing different states and rewards. However, although the existing TSC algorithms based on DRL have proposed many well-designed states and reward strategies, which combinations of states and rewards should be adopted in practice to achieve the performance margin of models remains a question that researchers are seeking the answer to. Therefore, we introduce a general simulation platform to test and compare experimental performance under different combinations of states and rewards. Specifically, we test and analyze the experimental effects under different combinations of multiple traffic states and rewards through various TSC methods with a set of unified model settings. We further design and test some new state representations and reward strategies based on more detailed traffic information. The test results show that when researchers design the state and reward, refining the traffic state like vehicle running condition and making the state and reward match can make the experimental performance better than other combinations in most cases. We hope these results have some implications for the state and reward choice when researchers conduct experiments on TSC problem or other traffic decision management problems.

Deep reinforcement learning for traffic signal control with consistent state and reward design approach

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Traffic Signal Control Using Hybrid Action Space Deep Reinforcement Learning

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Deep Reinforcement Learning for Adaptive Traffic Signal Control

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Optimizing Traffic Signal Control using High-Dimensional State Representation and Efficient Deep Reinforcement Learning

Diagnosing Reinforcement Learning for Traffic Signal Control

Using a Deep Reinforcement Learning Agent for Traffic Signal Control

A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

Guidelines for Parameter Selection in Traffic Light Control Methods Using Reinforcement Learning: Insights from Empirical Studies

Model-Based Deep Reinforcement Learning with Traffic Inference for Traffic Signal Control

Reinforcement Learning-Based Intelligent Traffic Signal Control Considering Sensing Information of Railway

A Deep Reinforcement Learning Approach for Fair Traffic Signal Control

RA-TSC: Learning Adaptive Traffic Signal Control Strategy Via Deep Reinforcement Learning

A Survey on Deep Learning-Based Traffic Signal Control