Abstract:Traffic signal control (TSC) plays a crucial role in enhancing traffic capacity. In recent years, researchers have demonstrated improved performance by utilizing deep reinforcement learning (DRL) for optimizing TSC. However, existing DRL frameworks predominantly rely on manually crafted states, actions, and reward designs, which limit direct information exchange between the DRL agent and the environment. To overcome this challenge, we propose a novel design method that maintains consistency among states, actions, and rewards, named uniformity state-action-reward (USAR) method for TSC. The USAR method relies on: 1) Updating the action selection for the next time step using a formula based on the state perceived by the agent at the current time step, thereby encouraging rapid convergence to the optimal strategy from state perception to action; and 2) integrating the state representation with the reward function design, allowing for precise assessment of the efficacy of past action strategies based on the received feedback rewards. The consistency-preserving design method jointly optimizes the TSC strategy through the updates and feedback among the Markov elements. Furthermore, the method proposed in this paper employs a residual block into the DRL model. It introduces an additional pathway between the input and output layers to transfer feature information, thus promoting the flow of information across different network layers. To assess the effectiveness of our approach, we conducted a series of simulation experiments using the simulation of urban mobility. The USAR method, incorporating a residual block, outperformed other methods and exhibited the best performance in several evaluation metrics.

Training for More Robust and Practical Adaptive Signal Control Models

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

Learning in practice: reinforcement learning-based traffic signal control augmented with actuated control

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

Traffic Signal Timing via Parallel Reinforcement Learning

A Stochastic Adaptive Traffic Signal Control Model Based on Fuzzy Reinforcement Learning

Adaptive Traffic Signal Control: Deep Reinforcement Learning Algorithm with Experience Replay and Target Network

Deep Reinforcement Learning for Traffic Signal Control Model and Adaptation Study

Multiphase Traffic Signal Adaptive Controlling Model for Urban Isolated Intersection

Optimization of Control Parameters for Adaptive Traffic-Actuated Signal Control

Adaptive Traffic Signal Control with Deep Reinforcement Learning An Exploratory Investigation

Deep Reinforcement Learning for Adaptive Traffic Signal Control

Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control

Training a Reinforcement Learning Agent with AutoRL for Traffic Signal Control

Lyapunov Function Consistent Adaptive Network Signal Control with Back Pressure and Reinforcement Learning

Training Reinforcement Learning Agent for Traffic Signal Control under Different Traffic Conditions

An Attention Reinforcement Learning–Based Strategy for Large-Scale Adaptive Traffic Signal Control System

A Deep Adaptive Traffic Signal Controller with Long-Term Planning Horizon and Spatial-Temporal State Definition under Dynamic Traffic Fluctuations

Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning

Environmental Adaptive Urban Traffic Signal Control Based on Reinforcement Learning Algorithm

Adaptive Traffic Signal Control With Deep Reinforcement Learning and High Dimensional Sensory Inputs: Case Study and Comprehensive Sensitivity Analyses