Optimizing Traffic Signal Control using High-Dimensional State Representation and Efficient Deep Reinforcement Learning

Lawrence Francis,Blessed Guda,Ahmed Biyabani

2024-11-12

Abstract:In reinforcement learning-based (RL-based) traffic signal control (TSC), decisions on the signal timing are made based on the available information on vehicles at a road intersection. This forms the state representation for the RL environment which can either be high-dimensional containing several variables or a low-dimensional vector. Current studies suggest that using high dimensional state representations does not lead to improved performance on TSC. However, we argue, with experimental results, that the use of high dimensional state representations can, in fact, lead to improved TSC performance with improvements up to 17.9% of the average waiting time. This high-dimensional representation is obtainable using the cost-effective vehicle-to-infrastructure (V2I) communication, encouraging its adoption for TSC. Additionally, given the large size of the state, we identified the need to have computational efficient models and explored model compression via pruning.

Systems and Control,Artificial Intelligence

What problem does this paper attempt to address?

This paper attempts to address the problem of how to optimize traffic signal control (TSC) using high-dimensional state representation and efficient deep reinforcement learning methods to reduce traffic congestion and improve traffic efficiency. Specifically, the paper discusses the following points: 1. **Effectiveness of High-Dimensional State Representation**: Current research generally believes that using high-dimensional state representation does not significantly improve the performance of traffic signal control. However, the authors demonstrate through experimental results that high-dimensional state representation can actually bring performance improvements, especially in terms of average waiting time, with a maximum improvement of 17.9%. 2. **Application of V2I Communication**: The paper proposes that through vehicle-to-infrastructure (V2I) communication technology, rich vehicle information such as location, speed, waiting time, and delay can be obtained at a low cost. This information can be used to construct high-dimensional state representation. 3. **Improvement of Computational Efficiency**: Since high-dimensional state representation increases the computational burden, the authors introduce model compression techniques (such as pruning) to reduce the number of model parameters, thereby lowering computational costs and energy consumption. In summary, the paper aims to demonstrate the effectiveness of high-dimensional state representation in traffic signal control and proposes a solution that combines V2I communication and model compression techniques to achieve more efficient and environmentally friendly traffic signal control.

Optimizing Traffic Signal Control using High-Dimensional State Representation and Efficient Deep Reinforcement Learning

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

Deep reinforcement learning for traffic signal control with consistent state and reward design approach

DNLight: Learning Efficient Evaluation for Traffic Signal Control

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

Adaptive Traffic Signal Control With Deep Reinforcement Learning and High Dimensional Sensory Inputs: Case Study and Comprehensive Sensitivity Analyses

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

Deep Reinforcement Learning for Adaptive Traffic Signal Control

Guidelines for Parameter Selection in Traffic Light Control Methods Using Reinforcement Learning: Insights from Empirical Studies

Traffic Signal Control Using Hybrid Action Space Deep Reinforcement Learning

Adaptive Transit Signal Priority based on Deep Reinforcement Learning and Connected Vehicles in a Traffic Microsimulation Environment

Scalable Traffic Signal Controls using Fog-Cloud Based Multiagent Reinforcement Learning

Learning in practice: reinforcement learning-based traffic signal control augmented with actuated control

Optimizing Traffic Signal Control in Mixed Traffic Scenarios: A Predictive Traffic Information-Based Deep Reinforcement Learning Approach

iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

A scalable approach to optimize traffic signal control with federated reinforcement learning

Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control

Leveraging Queue Length and Attention Mechanisms for Enhanced Traffic Signal Control Optimization