Adaptive urban traffic signal control based on enhanced deep reinforcement learning

Changjian Cai,Min Wei
DOI: https://doi.org/10.1038/s41598-024-64885-w
IF: 4.6
2024-06-21
Scientific Reports
Abstract:One of the focal points in the field of intelligent transportation is the intelligent control of traffic signals (TS), aimed at enhancing the efficiency of urban road networks through specific algorithms. Deep Reinforcement Learning (DRL) algorithms have become mainstream, yet they suffer from inefficient training sample selection, leading to slow convergence. Additionally, enhancing model robustness is crucial for adapting to diverse traffic conditions. Hence, this paper proposes an enhanced method for traffic signal control (TSC) based on DRL. This approach utilizes dueling network and double q-learning to alleviate the overestimation issue of DRL. Additionally, it introduces a priority sampling mechanism to enhance the utilization efficiency of samples in memory. Moreover, noise parameters are integrated into the neural network model during training to bolster its robustness. By representing high-dimensional real-time traffic information as matrices, and employing a phase-cycled action space to guide the decision-making of intelligent agents. Additionally, utilizing a reward function that closely mirrors real-world scenarios to guide model training. Experimental results demonstrate faster convergence and optimal performance in metrics such as queue length and waiting time. Testing experiments further validate the method's robustness across different traffic flow scenarios.
multidisciplinary sciences
What problem does this paper attempt to address?
### Problems Addressed by the Paper The paper primarily explores the issue of intelligent control in urban traffic signal control (TSC) and proposes an enhanced deep reinforcement learning-based method to address the shortcomings of existing methods. Specifically: 1. **Improving Training Efficiency**: Existing deep reinforcement learning (DRL) algorithms have inefficiencies in sample selection, leading to slow convergence. This paper introduces a prioritized experience replay mechanism to improve the utilization efficiency of samples in the memory pool, thereby accelerating the training speed. 2. **Enhancing Model Robustness**: To adapt to diverse traffic conditions, enhancing the robustness of the model is crucial. This paper improves the model's adaptability to different traffic flow scenarios by adding noise parameters during the neural network training process. 3. **Optimizing Traffic Signal Control Strategies**: The paper proposes a traffic signal control model (PN_D3QN) that combines dueling network, double Q-learning, and prioritized experience replay. The aim is to alleviate traffic congestion through these technical means. Experimental results show that the proposed method exhibits faster convergence speed and better performance metrics, such as queue length and waiting time, in various traffic flow scenarios. This demonstrates the effectiveness and robustness of the proposed method.