Abstract:Effective traffic control methods have great potential in alleviating network congestion. Particularly, in an urban network consisting of heterogeneous roads ( e.g., freeways and urban roads), how to integrate and coordinate control policies on different roads is a critical issue in large-scale networks. This study addresses this question from two aspects: modeling and control. From the modeling aspect, we formulate the hybrid traffic modeling in heterogeneous networks with the Asymmetric Cell Transmission Model (ACTM) for freeways and the generalized bathtub model for urban roads. For the control aspect, this study considers two representative control approaches: ramp metering for freeways and perimeter control for urban roads, and we aim to develop a deep reinforcement learning (DRL)-based coordinated control framework for large-scale networks. However, there are two significant challenges in the coordinated control in large-scale networks with DRL methods: non-stationary environment and large search space. To address both issues, we incorporate the demonstration to guide the DRL method for better convergence by introducing the concept of "teacher" and "student" models. The teacher models are traditional controllers that provide control demonstrations. For instance, ALINEA and Gating are two representative feedback controllers for ramp metering and perimeter control which can be "teacher" models. The student models are DRL methods, which learn from teachers and aim to surpass the teachers' performance. Additionally, we develop a parallel training scheme to accelerate the proposed DRL method. To validate the proposed framework, we conduct two case studies in a small-scale network and a real-world large-scale traffic network in Hong Kong. Numerical results show that the proposed DRL method outperforms demonstrators as well as DRL methods, and the coordinated control is more effective than just controlling ramps or perimeters respectively. The research outcome reveals the great potential of combining traditional controllers with DRL for coordinated control in large-scale networks.

A Highway Entrance Ramp Control Method Based on Deep Reinforcement Learning

Deep Reinforcement Learning Based Lane-Level Variable Speed Limit Control

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

Freeway network traffic management based on distributed reinforcement learning

A Cyber-Physical System for Freeway Ramp Meter Signal Control Using Deep Reinforcement Learning in a Connected Environment

A Freeway Traffic Flow Control Model Based on Distributed Reinforcement Learning

A Deep Reinforcement Learning Approach for Ramp Metering Based on Traffic Video Data

An Indirect Reinforcement Learning Approach For Ramp Control Under Incident-Induced Congestion

Integrated Traffic Control for Freeway Recurrent Bottleneck Based on Deep Reinforcement Learning

Investigating Autonomous Vehicle Driving Strategies in Highway Ramp Merging Zones

Demonstration-guided deep reinforcement learning for coordinated ramp metering and perimeter control in large scale networks

Autonomous Highway Merging in Mixed Traffic Using Reinforcement Learning and Motion Predictive Safety Controller

Efficiency and equity based freeway traffic network flow control

An Optimized On-Ramp Metering Method for Urban Expressway Based on Reinforcement Learning.

A Deep Reinforcement Learning Approach for Automated On-Ramp Merging

Enhancing reinforcement learning‐based ramp metering performance at freeway uncertain bottlenecks using curriculum learning

Reinforcement Learning for Ramp Control: an Analysis of Learning Parameters

Autonomous On-ramp Merge Strategy Using Deep Reinforcement Learning in Uncertain Highway Environment

Intelligent Ramp Control for Incident Response Using Dyna-Q Architecture

Indirect Reinforcement Learning for Incident-Responsive Ramp Control