Abstract:Traffic signal control (TSC) plays a crucial role in enhancing traffic capacity. In recent years, researchers have demonstrated improved performance by utilizing deep reinforcement learning (DRL) for optimizing TSC. However, existing DRL frameworks predominantly rely on manually crafted states, actions, and reward designs, which limit direct information exchange between the DRL agent and the environment. To overcome this challenge, we propose a novel design method that maintains consistency among states, actions, and rewards, named uniformity state-action-reward (USAR) method for TSC. The USAR method relies on: 1) Updating the action selection for the next time step using a formula based on the state perceived by the agent at the current time step, thereby encouraging rapid convergence to the optimal strategy from state perception to action; and 2) integrating the state representation with the reward function design, allowing for precise assessment of the efficacy of past action strategies based on the received feedback rewards. The consistency-preserving design method jointly optimizes the TSC strategy through the updates and feedback among the Markov elements. Furthermore, the method proposed in this paper employs a residual block into the DRL model. It introduces an additional pathway between the input and output layers to transfer feature information, thus promoting the flow of information across different network layers. To assess the effectiveness of our approach, we conducted a series of simulation experiments using the simulation of urban mobility. The USAR method, incorporating a residual block, outperformed other methods and exhibited the best performance in several evaluation metrics.

MonitorLight: Reinforcement Learning-based Traffic Signal Control Using Mixed Pressure Monitoring

MonitorLight

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

DNLight: Learning Efficient Evaluation for Traffic Signal Control

IPDALight: Intensity- and Phase Duration-Aware Traffic Signal Control Based on Reinforcement Learning

DynamicLight: Dynamically Tuning Traffic Signal Duration with DRL

Efficient Pressure: Improving efficiency for signalized intersections

MTLight: Efficient Multi-Task Reinforcement Learning for Traffic Signal Control

PhaseLight: an Universal and Practical Traffic Signal Control Algorithms Based on Reinforcement Learning

DynamicLight: Two-Stage Dynamic Traffic Signal Timing

PDLight: A Deep Reinforcement Learning Traffic Light Control Algorithm with Pressure and Dynamic Light Duration

Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control

AttentionLight: Rethinking queue length and attention mechanism for traffic signal control

DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback.

Leveraging Queue Length and Attention Mechanisms for Enhanced Traffic Signal Control Optimization

ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control

Traffic Signal Control in Mixed Traffic Environment Based on Advance Decision and Reinforcement Learning

FairLight: Fairness-Aware Autonomous Traffic Signal Control with Hierarchical Action Space

PressLight

Brief Industry Paper: RTLight: Digital Twin-Based Real-Time Federated Traffic Signal Control

Adaptive urban traffic signal control based on enhanced deep reinforcement learning