Abstract:Effective traffic control methods have great potential in alleviating network congestion. Particularly, in an urban network consisting of heterogeneous roads ( e.g., freeways and urban roads), how to integrate and coordinate control policies on different roads is a critical issue in large-scale networks. This study addresses this question from two aspects: modeling and control. From the modeling aspect, we formulate the hybrid traffic modeling in heterogeneous networks with the Asymmetric Cell Transmission Model (ACTM) for freeways and the generalized bathtub model for urban roads. For the control aspect, this study considers two representative control approaches: ramp metering for freeways and perimeter control for urban roads, and we aim to develop a deep reinforcement learning (DRL)-based coordinated control framework for large-scale networks. However, there are two significant challenges in the coordinated control in large-scale networks with DRL methods: non-stationary environment and large search space. To address both issues, we incorporate the demonstration to guide the DRL method for better convergence by introducing the concept of "teacher" and "student" models. The teacher models are traditional controllers that provide control demonstrations. For instance, ALINEA and Gating are two representative feedback controllers for ramp metering and perimeter control which can be "teacher" models. The student models are DRL methods, which learn from teachers and aim to surpass the teachers' performance. Additionally, we develop a parallel training scheme to accelerate the proposed DRL method. To validate the proposed framework, we conduct two case studies in a small-scale network and a real-world large-scale traffic network in Hong Kong. Numerical results show that the proposed DRL method outperforms demonstrators as well as DRL methods, and the coordinated control is more effective than just controlling ramps or perimeters respectively. The research outcome reveals the great potential of combining traditional controllers with DRL for coordinated control in large-scale networks.

A Hierarchical Framework for Passenger Inflow Control in Metro System With Reinforcement Learning

Dynamic flow control model and algorithm for metro network under FIFO condition

Data-driven timetable design and passenger flow control optimization in metro lines

Integrated train timetabling and rolling stock rescheduling for a disturbed metro system: A hybrid deep reinforcement learning and adaptive large neighborhood search approach

A Collaborative Optimization Method for Train Scheduling and Passenger Flow Assignment Based on Multi-Agent Reinforcement Learning

A Passenger Flow Control Method for Subway Network Based on Network Controllability

Cooperative passenger flow control in an oversaturated metro network with operational risk thresholds

IPF-HMGNN: A novel integrative prediction framework for metro passenger flow

Integrated optimization of train services plan and passenger flow control on an oversaturated suburban metro line

Robust collaborative passenger flow control on a congested metro line: A joint optimization with train timetabling

Simulation-based Adaptive Optimization for Passenger Flow Control Measures at Metro Stations

Service-oriented train timetabling with collaborative passenger flow control on an oversaturated metro line: An integer linear optimization approach

Demonstration-guided deep reinforcement learning for coordinated ramp metering and perimeter control in large scale networks

Integrated Traffic Control for Freeway Recurrent Bottleneck Based on Deep Reinforcement Learning

An integrated optimization approach for passenger flow control strategy and metro train scheduling considering skip-stop patterns in special situations

Integrated optimization model for hierarchical service network design and passenger assignment in an urban rail transit network: A Lagrangian duality reformulation and an iterative layered optimization framework based on forward-passing and backpropagation

Collaborative optimization of passenger flow control and bus-bridging services in commuting metro lines

A Novel Framework Combining MPC and Deep Reinforcement Learning With Application to Freeway Traffic Control

A modeling framework to integrate frequency - and schedule-based passenger assignment approaches for coordinated path choice and space-time trajectory estimation based on multi-source observations

A cloud model-based method for passenger flow control at subway stations: A real-world case study