Abstract:Recent advances in deep reinforcement learning have shown promising results in solving sophisticated control problems with high dimensional states and action space. Inspired by this, we use the latest deep reinforcement learning (DRL) methods to improve freeway traffic mobility and alleviate recurring bottlenecks and congestion. More specifically, this paper proposes a centralized traffic control system that can coordinate multiple ramp metering (RM) and variable speed limit (VSL) traffic controllers on freeways to minimize the total travel time. The system uses a novel double-layer structure to synchronize different traffic controllers and introduces the actor-critic-based DRL methods to learn joint actions in a high-dimensional traffic environment. The reward function takes into account the waiting time of vehicles, the average speed of different road sections, and the on-ramp queuing limit to improve traffic mobility. We also proposed an integrated feedback controller as a benchmark. The simulation results show that the actor-critic-based methods are superior to other methods and can save more than 20% of the total travel time. We also analyzed the curse of dimensionality problem by comparing the performance of two scenarios in the simulation: one is a single-ramp interweaving area scenario; the other is a large freeway corridor with multiple on-ramps and off-ramps. The results show that our system can effectively handle these two situations without significant performance degradation, which means that the centralized control system can effectively control freeway corridors by directly guiding various traffic controllers. This also leads to the conclusion that we can use a centralized actor-critic-based control unit to manage medium-scale freeway traffic to save computing resources instead of using complex collaboration strategies.

Coordinated Variable Speed Limit Control for Consecutive Bottlenecks on Freeways Using Multiagent Reinforcement Learning

Deep Reinforcement Learning Based Lane-Level Variable Speed Limit Control

Multi-Agent Deep Reinforcement Learning for Multi-Lane Freeways Differential Variable Speed Limit Control in Mixed Traffic Environment

Reinforcement Learning-Based Variable Speed Limit Control Strategy to Reduce Traffic Congestion at Freeway Recurrent Bottlenecks

MARVEL: Bringing Multi-Agent Reinforcement- Learning Based Variable Speed Limit Controllers Closer to Deployment

Variable Speed Limit Control at Freeway Merge Bottlenecks Based on Reinforcement Learning 0

Variable Speed Limit Control for the Motorway–Urban Merging Bottlenecks Using Multi-Agent Reinforcement Learning

MARVEL: Multi-Agent Reinforcement-Learning for Large-Scale Variable Speed Limits

A New Reinforcement Learning-Based Variable Speed Limit Control Approach to Improve Traffic Efficiency Against Freeway Jam Waves

A variable speed limit control approach for freeway tunnels based on the model-based reinforcement learning framework with safety perception

Variable Speed Limit Control Design for Relieving Congestion Caused by Active Bottlenecks

Differential Variable Speed Limits Control for Freeway Recurrent Bottlenecks via Deep Reinforcement learning

A New Solution for Freeway Congestion: Cooperative Speed Limit Control Using Distributed Reinforcement Learning

Integrated Traffic Control for Freeway Recurrent Bottleneck Based on Deep Reinforcement Learning

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

Integrated Variable Speed Limits and Lane-Changing Control for Freeway Lane-Drop Bottlenecks

Enhancing multi-scenario applicability of freeway variable speed limit control strategies using continual learning

Cycle-based Variable Speed Limit Methodology for Improved Freeway Merging

Field Deployment of Multi-Agent Reinforcement Learning Based Variable Speed Limit Controllers

Variable Speed Limit Strategy to Improve the Safety and Environmental Impact of Freeway Traffic

Variable Speed Limit Modelling to Improve Traffic Safety and Efficiency of Mixed Traffic Flow by a Two-Stage Framework