Abstract:Abstract How to design a good traffic signal switch scheme is crucial to alleviate transportation pressure. However, some existing traffic signal control methods based on reinforcement learning apply a convolutional neural network (CNN) to extract features from the state matrix, which is too coarse because state matrices sometimes are sparse. These methods use the discrete traffic state encoding (DTSE) approach to build the position and speed matrices. We overcome this problem in two ways: by reducing the number of zero value in the state matrix and using a spatial attention mechanism in convolution operations. This paper redefines the state at the intersection by dividing each entering road with an unequal length and then concatenating them. Our division method can reduce the state matrix’s sparsity by reducing the size of the input matrix and the number of zero value. We then propose a deep Q-learning network (DQN) with spatial attention in CNN to control the traffic light. The spatial attention mechanism can focus on the specific element in the con-volutional feature vectors and overcome the sparse matrix’s difficult problem to extract features. Besides, we set a discount factor β on the reward definition, which can diminish the received reward and alleviate the overestimation caused by DQN. We conduct experiments on the Simulation of Urban MObility (SUMO) with two datasets, a synthetic intersection and Cologne’s traffic intersection in Germany. Experimental results on synthetic intersection demonstrate that our method can reduce delay, travel time, and queue length than three other methods. Results on Cologne confirm that the proposed method can save 23.7%, 9.1%, 4.5% travel time compared with the fixed-time traffic light control, self-organizing traffic light control, and DQN method, respectively.

AttentionLight: Rethinking queue length and attention mechanism for traffic signal control

Leveraging Queue Length and Attention Mechanisms for Enhanced Traffic Signal Control Optimization

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control

MonitorLight: Reinforcement Learning-based Traffic Signal Control Using Mixed Pressure Monitoring

MTLight: Efficient Multi-Task Reinforcement Learning for Traffic Signal Control

PhaseLight: an Universal and Practical Traffic Signal Control Algorithms Based on Reinforcement Learning

Efficient Pressure: Improving efficiency for signalized intersections

IPDALight: Intensity- and Phase Duration-Aware Traffic Signal Control Based on Reinforcement Learning

Expression is Enough: Improving Traffic Signal Control with Advanced Traffic State Representation.

MonitorLight

DynamicLight: Dynamically Tuning Traffic Signal Duration with DRL

FairLight: Fairness-Aware Autonomous Traffic Signal Control with Hierarchical Action Space

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Intelligent Traffic Light Switch Based on Deep Reinforcement Learning with Attention Mechanism

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback.

Guidelines for Parameter Selection in Traffic Light Control Methods Using Reinforcement Learning: Insights from Empirical Studies

DynamicLight: Two-Stage Dynamic Traffic Signal Timing

iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement