Abstract:Recent advances in deep reinforcement learning have shown promising results in solving sophisticated control problems with high dimensional states and action space. Inspired by this, we use the latest deep reinforcement learning (DRL) methods to improve freeway traffic mobility and alleviate recurring bottlenecks and congestion. More specifically, this paper proposes a centralized traffic control system that can coordinate multiple ramp metering (RM) and variable speed limit (VSL) traffic controllers on freeways to minimize the total travel time. The system uses a novel double-layer structure to synchronize different traffic controllers and introduces the actor-critic-based DRL methods to learn joint actions in a high-dimensional traffic environment. The reward function takes into account the waiting time of vehicles, the average speed of different road sections, and the on-ramp queuing limit to improve traffic mobility. We also proposed an integrated feedback controller as a benchmark. The simulation results show that the actor-critic-based methods are superior to other methods and can save more than 20% of the total travel time. We also analyzed the curse of dimensionality problem by comparing the performance of two scenarios in the simulation: one is a single-ramp interweaving area scenario; the other is a large freeway corridor with multiple on-ramps and off-ramps. The results show that our system can effectively handle these two situations without significant performance degradation, which means that the centralized control system can effectively control freeway corridors by directly guiding various traffic controllers. This also leads to the conclusion that we can use a centralized actor-critic-based control unit to manage medium-scale freeway traffic to save computing resources instead of using complex collaboration strategies.

Freeway network traffic management based on distributed reinforcement learning

Efficiency and equity based freeway traffic network flow control

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

A Freeway Traffic Flow Control Model Based on Distributed Reinforcement Learning

Integrated Traffic Control for Freeway Recurrent Bottleneck Based on Deep Reinforcement Learning

Demonstration-guided Deep Reinforcement Learning for Coordinated Ramp Metering and Perimeter Control in Large Scale Networks

A New Solution for Freeway Congestion: Cooperative Speed Limit Control Using Distributed Reinforcement Learning

A Cyber-Physical System for Freeway Ramp Meter Signal Control Using Deep Reinforcement Learning in a Connected Environment

Differential Variable Speed Limits Control for Freeway Recurrent Bottlenecks via Deep Reinforcement learning

Deep Reinforcement Learning to Maximize Arterial Usage during Extreme Congestion

A Highway Entrance Ramp Control Method Based on Deep Reinforcement Learning

A multi‐agent deep reinforcement learning approach for traffic signal coordination

A Novel Framework Combining MPC and Deep Reinforcement Learning With Application to Freeway Traffic Control

Coordinated Ramp Metering with Equity Consideration Using Reinforcement Learning

Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning

Freeway congestion management on multiple consecutive bottlenecks with RL‐based headway control of autonomous vehicles

DVS-RG: Differential Variable Speed Limits Control using Deep Reinforcement Learning with Graph State Representation

A Comparative Analysis of Deep Reinforcement Learning-enabled Freeway Decision-making for Automated Vehicles