Abstract:Traffic signal control (TSC) plays a crucial role in enhancing traffic capacity. In recent years, researchers have demonstrated improved performance by utilizing deep reinforcement learning (DRL) for optimizing TSC. However, existing DRL frameworks predominantly rely on manually crafted states, actions, and reward designs, which limit direct information exchange between the DRL agent and the environment. To overcome this challenge, we propose a novel design method that maintains consistency among states, actions, and rewards, named uniformity state-action-reward (USAR) method for TSC. The USAR method relies on: 1) Updating the action selection for the next time step using a formula based on the state perceived by the agent at the current time step, thereby encouraging rapid convergence to the optimal strategy from state perception to action; and 2) integrating the state representation with the reward function design, allowing for precise assessment of the efficacy of past action strategies based on the received feedback rewards. The consistency-preserving design method jointly optimizes the TSC strategy through the updates and feedback among the Markov elements. Furthermore, the method proposed in this paper employs a residual block into the DRL model. It introduces an additional pathway between the input and output layers to transfer feature information, thus promoting the flow of information across different network layers. To assess the effectiveness of our approach, we conducted a series of simulation experiments using the simulation of urban mobility. The USAR method, incorporating a residual block, outperformed other methods and exhibited the best performance in several evaluation metrics.

Cooperative Traffic Signal Control Based on Biased ReLU Neural Network Approximation

Multi-agent Reinforcement Traffic Signal Control based on Interpretable Influence Mechanism and Biased ReLU Approximation

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Biased Pressure: Cyclic Reinforcement Learning Model for Intelligent Traffic Signal Control

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

A multi‐agent deep reinforcement learning approach for traffic signal coordination

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

Cooperative Reinforcement Learning on Traffic Signal Control

Traffic Signal Timing via Parallel Reinforcement Learning

A Novel Multi-Agent Deep RL Approach for Traffic Signal Control

Hierarchically and Cooperatively Learning Traffic Signal Control

Cooperative Control for Multi-Intersection Traffic Signal Based on Deep Reinforcement Learning and Imitation Learning

Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network

Mean Field Multi-Agent Reinforcement Learning Method for Area Traffic Signal Control

Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach

Lyapunov Function Consistent Adaptive Network Signal Control with Back Pressure and Reinforcement Learning

Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning

Traffic Signal Control Optimization Based on Neural Network in the Framework of Model Predictive Control