Abstract:Recent advances in deep reinforcement learning have shown promising results in solving sophisticated control problems with high dimensional states and action space. Inspired by this, we use the latest deep reinforcement learning (DRL) methods to improve freeway traffic mobility and alleviate recurring bottlenecks and congestion. More specifically, this paper proposes a centralized traffic control system that can coordinate multiple ramp metering (RM) and variable speed limit (VSL) traffic controllers on freeways to minimize the total travel time. The system uses a novel double-layer structure to synchronize different traffic controllers and introduces the actor-critic-based DRL methods to learn joint actions in a high-dimensional traffic environment. The reward function takes into account the waiting time of vehicles, the average speed of different road sections, and the on-ramp queuing limit to improve traffic mobility. We also proposed an integrated feedback controller as a benchmark. The simulation results show that the actor-critic-based methods are superior to other methods and can save more than 20% of the total travel time. We also analyzed the curse of dimensionality problem by comparing the performance of two scenarios in the simulation: one is a single-ramp interweaving area scenario; the other is a large freeway corridor with multiple on-ramps and off-ramps. The results show that our system can effectively handle these two situations without significant performance degradation, which means that the centralized control system can effectively control freeway corridors by directly guiding various traffic controllers. This also leads to the conclusion that we can use a centralized actor-critic-based control unit to manage medium-scale freeway traffic to save computing resources instead of using complex collaboration strategies.

ZiXia: A Reinforcement Learning Approach via Adjusted Ranking Reward for Internet Congestion Control

A Multi-objective Reinforcement Learning Perspective on Internet Congestion Control

TRCC: Transferable Congestion Control with Reinforcement Learning

Deep Reinforcement Learning for Smart City Communication Networks

RAN Information-Assisted TCP Congestion Control Using Deep Reinforcement Learning with Reward Redistribution

Reinforcement Learning Based Congestion Control in a Real Environment.

RAN Information-assisted TCP Congestion Control via DRL with Reward Redistribution

An Intelligent Congestion Control Strategy in Heterogeneous V2X Based on Deep Reinforcement Learning

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Optimizing Communication in Deep Reinforcement Learning with XingTian

Integrated Traffic Control for Freeway Recurrent Bottleneck Based on Deep Reinforcement Learning

DECC: Achieving Low Latency in Data Center Networks with Deep Reinforcement Learning

Dynamic TCP Initial Windows and Congestion Control Schemes Through Reinforcement Learning

Deep Reinforcement Learning for Router Selection in Network with Heavy Traffic

Internet Congestion Control via Deep Reinforcement Learning

DeepCC: Bridging the Gap Between Congestion Control and Applications Via Multiobjective Optimization

A Reinforcement Learning-Based Traffic Engineering Algorithm for Enterprise Network Backbone Links

DeepCC: Bridging the Gap Between Congestion Control and Applications via Multi-Objective Optimization

Making TCP BBR Pacing Adaptive with Domain Knowledge Assisted Reinforcement Learning

Optimizing Traffic Efficiency Via a Reinforcement Learning Approach Based on Time Allocation

A Deep Reinforcement Learning Framework for Optimizing Congestion Control in Data Centers