Abstract:Effective traffic control methods have great potential in alleviating network congestion. Particularly, in an urban network consisting of heterogeneous roads ( e.g., freeways and urban roads), how to integrate and coordinate control policies on different roads is a critical issue in large-scale networks. This study addresses this question from two aspects: modeling and control. From the modeling aspect, we formulate the hybrid traffic modeling in heterogeneous networks with the Asymmetric Cell Transmission Model (ACTM) for freeways and the generalized bathtub model for urban roads. For the control aspect, this study considers two representative control approaches: ramp metering for freeways and perimeter control for urban roads, and we aim to develop a deep reinforcement learning (DRL)-based coordinated control framework for large-scale networks. However, there are two significant challenges in the coordinated control in large-scale networks with DRL methods: non-stationary environment and large search space. To address both issues, we incorporate the demonstration to guide the DRL method for better convergence by introducing the concept of "teacher" and "student" models. The teacher models are traditional controllers that provide control demonstrations. For instance, ALINEA and Gating are two representative feedback controllers for ramp metering and perimeter control which can be "teacher" models. The student models are DRL methods, which learn from teachers and aim to surpass the teachers' performance. Additionally, we develop a parallel training scheme to accelerate the proposed DRL method. To validate the proposed framework, we conduct two case studies in a small-scale network and a real-world large-scale traffic network in Hong Kong. Numerical results show that the proposed DRL method outperforms demonstrators as well as DRL methods, and the coordinated control is more effective than just controlling ramps or perimeters respectively. The research outcome reveals the great potential of combining traditional controllers with DRL for coordinated control in large-scale networks.

DHP Method for Ramp Metering of Freeway Traffic

When to Control the Ramps on Freeway Corridors? A Novel Stability-and-MFD-Based Approach

Local Ramp Metering in Random-Location Bottlenecks Downstream of Metered On-Ramp

Local Ramp Metering in the Case of Distant Downstream Bottlenecks

Integrated Control of Internal Boundary and Ramp Inflows for Lane-free Traffic of Automated Vehicles on Freeways.

An Adaptive Control Scheme for Local and Coordinated Ramp Metering

Local Ramp Metering with Distant Downstream Bottlenecks: A Comparative Study.

Improved Bottleneck Approach for Heuristic Systemwide Responsive Ramp Metering on Congested Freeways

Adaptive Control of On-Ramp Metering for Highway Based on Genetic-Fuzzy Approach

Adaptive Ramp Metering Control for Urban Freeway Using Large-Scale Data

Perimeter Control-Based Coordinated Ramp Metering with Linked Queue Management for Freeway Networks

Control Priorities of Coordinated On-Ramps Metering Based on Quantitative Hierarchical Model

An Extended Model Predictive Control Approach To Coordinated Ramp Metering

Coordinated Ramp Metering Considering the Dynamics of Mixed-Autonomy Traffic

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

An Integrated Approach for Dynamic Traffic Routing and Ramp Metering Using Sliding Mode Control

Method for Freeway On-ramp Control Based on Fuzzy Logic

Design and Simulation of Coordinated Ramp Control System on Freeway

Demonstration-guided deep reinforcement learning for coordinated ramp metering and perimeter control in large scale networks

Freeway Ramp Metering Using Fuzzy Logic and Genetic Proportional Plus Integral Control

An Indicator for Determining the Control Timing of Ramp Metering on Urban Freeway Corridors