Abstract:Abstract How to design a good traffic signal switch scheme is crucial to alleviate transportation pressure. However, some existing traffic signal control methods based on reinforcement learning apply a convolutional neural network (CNN) to extract features from the state matrix, which is too coarse because state matrices sometimes are sparse. These methods use the discrete traffic state encoding (DTSE) approach to build the position and speed matrices. We overcome this problem in two ways: by reducing the number of zero value in the state matrix and using a spatial attention mechanism in convolution operations. This paper redefines the state at the intersection by dividing each entering road with an unequal length and then concatenating them. Our division method can reduce the state matrix’s sparsity by reducing the size of the input matrix and the number of zero value. We then propose a deep Q-learning network (DQN) with spatial attention in CNN to control the traffic light. The spatial attention mechanism can focus on the specific element in the con-volutional feature vectors and overcome the sparse matrix’s difficult problem to extract features. Besides, we set a discount factor β on the reward definition, which can diminish the received reward and alleviate the overestimation caused by DQN. We conduct experiments on the Simulation of Urban MObility (SUMO) with two datasets, a synthetic intersection and Cologne’s traffic intersection in Germany. Experimental results on synthetic intersection demonstrate that our method can reduce delay, travel time, and queue length than three other methods. Results on Cologne confirm that the proposed method can save 23.7%, 9.1%, 4.5% travel time compared with the fixed-time traffic light control, self-organizing traffic light control, and DQN method, respectively.

Intelligent Traffic Signal Control Based on Reinforcement Learning with State Reduction for Smart Cities

Q-learning Based Traffic Signal Control Method for an Isolated Intersection

Multiphase Traffic Signal Adaptive Controlling Model for Urban Isolated Intersection

A Stochastic Adaptive Control Model for Isolated Intersections

A Stochastic Adaptive Traffic Signal Control Model Based on Fuzzy Reinforcement Learning

Evaluation and Application of Urban Traffic Signal Optimizing Control Strategy Based on Reinforcement Learning

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning

A Method of Reinforcement Learning Based Automatic Traffic Signal Control

Deep Reinforcement Learning for Intersection Signal Control Considering Pedestrian Behavior

Carbon Dioxide Emission Reduction-Oriented Optimal Control of Traffic Signals in Mixed Traffic Flow Based on Deep Reinforcement Learning

Optimizing Traffic Efficiency Via a Reinforcement Learning Approach Based on Time Allocation

Research on Intelligent Signal Timing Optimization of Signalized Intersection Based on Deep Reinforcement Learning Using Floating Car Data

An Intelligent Traffic Signal Control System Based on Deep Reinforcement Learning

Intelligent Control Scheme for Traffic Signals at Multi-intersections based on Pedestrian Safety

A Deep Adaptive Traffic Signal Controller with Long-Term Planning Horizon and Spatial-Temporal State Definition under Dynamic Traffic Fluctuations

Intelligent Traffic Light Switch Based on Deep Reinforcement Learning with Attention Mechanism

A Traffic Signal Control Method Based on Asynchronous Reinforcement Learning