Abstract:Abstract How to design a good traffic signal switch scheme is crucial to alleviate transportation pressure. However, some existing traffic signal control methods based on reinforcement learning apply a convolutional neural network (CNN) to extract features from the state matrix, which is too coarse because state matrices sometimes are sparse. These methods use the discrete traffic state encoding (DTSE) approach to build the position and speed matrices. We overcome this problem in two ways: by reducing the number of zero value in the state matrix and using a spatial attention mechanism in convolution operations. This paper redefines the state at the intersection by dividing each entering road with an unequal length and then concatenating them. Our division method can reduce the state matrix’s sparsity by reducing the size of the input matrix and the number of zero value. We then propose a deep Q-learning network (DQN) with spatial attention in CNN to control the traffic light. The spatial attention mechanism can focus on the specific element in the con-volutional feature vectors and overcome the sparse matrix’s difficult problem to extract features. Besides, we set a discount factor β on the reward definition, which can diminish the received reward and alleviate the overestimation caused by DQN. We conduct experiments on the Simulation of Urban MObility (SUMO) with two datasets, a synthetic intersection and Cologne’s traffic intersection in Germany. Experimental results on synthetic intersection demonstrate that our method can reduce delay, travel time, and queue length than three other methods. Results on Cologne confirm that the proposed method can save 23.7%, 9.1%, 4.5% travel time compared with the fixed-time traffic light control, self-organizing traffic light control, and DQN method, respectively.

Imitation Learning Based Deep Reinforcement Learning for Traffic Signal Control

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Model-Based Deep Reinforcement Learning with Traffic Inference for Traffic Signal Control

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

Deep Reinforcement Learning for Adaptive Traffic Signal Control

Traffic Signal Timing via Parallel Reinforcement Learning

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Smarter and Safer Traffic Signal Controlling Via Deep Reinforcement Learning

Diagnosing Reinforcement Learning for Traffic Signal Control

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

Intelligent Traffic Light Switch Based on Deep Reinforcement Learning with Attention Mechanism

A multi‐agent deep reinforcement learning approach for traffic signal coordination

Reinforcement Learning Approaches for Traffic Signal Control under Missing Data

Hierarchically and Cooperatively Learning Traffic Signal Control

Random Ensemble Reinforcement Learning for Traffic Signal Control

Deep Reinforcement Learning for Traffic Signal Control Model and Adaptation Study

Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems

Urban Traffic Signal Control with Reinforcement Learning from Demonstration Data

A deep reinforcement learning network for traffic light cycle control