Abstract:Abstract How to design a good traffic signal switch scheme is crucial to alleviate transportation pressure. However, some existing traffic signal control methods based on reinforcement learning apply a convolutional neural network (CNN) to extract features from the state matrix, which is too coarse because state matrices sometimes are sparse. These methods use the discrete traffic state encoding (DTSE) approach to build the position and speed matrices. We overcome this problem in two ways: by reducing the number of zero value in the state matrix and using a spatial attention mechanism in convolution operations. This paper redefines the state at the intersection by dividing each entering road with an unequal length and then concatenating them. Our division method can reduce the state matrix’s sparsity by reducing the size of the input matrix and the number of zero value. We then propose a deep Q-learning network (DQN) with spatial attention in CNN to control the traffic light. The spatial attention mechanism can focus on the specific element in the con-volutional feature vectors and overcome the sparse matrix’s difficult problem to extract features. Besides, we set a discount factor β on the reward definition, which can diminish the received reward and alleviate the overestimation caused by DQN. We conduct experiments on the Simulation of Urban MObility (SUMO) with two datasets, a synthetic intersection and Cologne’s traffic intersection in Germany. Experimental results on synthetic intersection demonstrate that our method can reduce delay, travel time, and queue length than three other methods. Results on Cologne confirm that the proposed method can save 23.7%, 9.1%, 4.5% travel time compared with the fixed-time traffic light control, self-organizing traffic light control, and DQN method, respectively.

A deep reinforcement learning network for traffic light cycle control

Deep Reinforcement Learning for Traffic Light Control in Vehicular Networks

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

Researches on Intelligent Traffic Signal Control Based on Deep Reinforcement Learning

Traffic light control with reinforcement learning

Deep Reinforcement Learning for Traffic Signal Control Model and Adaptation Study

Intelligent Traffic Light Switch Based on Deep Reinforcement Learning with Attention Mechanism

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Adaptive Traffic Signal Control with Deep Reinforcement Learning An Exploratory Investigation

Hierarchically and Cooperatively Learning Traffic Signal Control

Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning

Training Reinforcement Learning Agent for Traffic Signal Control under Different Traffic Conditions

Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems

Imitation Learning Based Deep Reinforcement Learning for Traffic Signal Control

Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control

IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control.

Model-Based Deep Reinforcement Learning with Traffic Inference for Traffic Signal Control

Using a Deep Reinforcement Learning Agent for Traffic Signal Control

A distributed deep reinforcement learning method for traffic light control

Fairness Control of Traffic Light Via Deep Reinforcement Learning