Abstract:Effective traffic control methods have great potential in alleviating network congestion. Particularly, in an urban network consisting of heterogeneous roads ( e.g., freeways and urban roads), how to integrate and coordinate control policies on different roads is a critical issue in large-scale networks. This study addresses this question from two aspects: modeling and control. From the modeling aspect, we formulate the hybrid traffic modeling in heterogeneous networks with the Asymmetric Cell Transmission Model (ACTM) for freeways and the generalized bathtub model for urban roads. For the control aspect, this study considers two representative control approaches: ramp metering for freeways and perimeter control for urban roads, and we aim to develop a deep reinforcement learning (DRL)-based coordinated control framework for large-scale networks. However, there are two significant challenges in the coordinated control in large-scale networks with DRL methods: non-stationary environment and large search space. To address both issues, we incorporate the demonstration to guide the DRL method for better convergence by introducing the concept of "teacher" and "student" models. The teacher models are traditional controllers that provide control demonstrations. For instance, ALINEA and Gating are two representative feedback controllers for ramp metering and perimeter control which can be "teacher" models. The student models are DRL methods, which learn from teachers and aim to surpass the teachers' performance. Additionally, we develop a parallel training scheme to accelerate the proposed DRL method. To validate the proposed framework, we conduct two case studies in a small-scale network and a real-world large-scale traffic network in Hong Kong. Numerical results show that the proposed DRL method outperforms demonstrators as well as DRL methods, and the coordinated control is more effective than just controlling ramps or perimeters respectively. The research outcome reveals the great potential of combining traditional controllers with DRL for coordinated control in large-scale networks.

A Novel Distributed Reinforcement Learning Training System for Network Congestion Control

TRCC: Transferable Congestion Control with Reinforcement Learning

A Multi-objective Reinforcement Learning Perspective on Internet Congestion Control

Deep Reinforcement Learning for Smart City Communication Networks

C2: Co-design of Robots Via Concurrent-Network Coupling Online and Offline Reinforcement Learning

Internet Congestion Control via Deep Reinforcement Learning

Reinforcement Learning Based Congestion Control in a Real Environment.

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

A Deep Reinforcement Learning Framework for Optimizing Congestion Control in Data Centers

A Reinforcement Learning Congestion Control Algorithm for Smart Grid Networks

Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs

RAN Information-Assisted TCP Congestion Control Using Deep Reinforcement Learning with Reward Redistribution

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

ACC-RL: Adaptive Congestion Control Based on Reinforcement Learning in Power Distribution Networks with Data Centers

Congestion Control in SDN-Based Networks via Multi-Task Deep Reinforcement Learning

ZiXia: A Reinforcement Learning Approach via Adjusted Ranking Reward for Internet Congestion Control

Pareto: Fair Congestion Control With Online Reinforcement Learning

A distributed deep reinforcement learning method for traffic light control

DECC: Achieving Low Latency in Data Center Networks with Deep Reinforcement Learning

Demonstration-guided deep reinforcement learning for coordinated ramp metering and perimeter control in large scale networks