Abstract:This study proposed a novel approach called deep reinforcement learning with curriculum learning (DRLCL) to improve ramp metering efficacy under uncertain bottleneck conditions. The curriculum learning method transfers an optimal control policy from a simple on‐ramp bottleneck case to more challenging bottleneck tasks, while DRLCL agents explore and learn from the tasks gradually. Four RM control tasks were developed in the modified cell transmission model, including typical on‐ramp bottleneck, fixed downstream bottleneck, random‐location bottleneck, and multiple bottlenecks. Most current RM approaches are developed for fixed bottlenecks. However, the number and locations of bottlenecks are usually uncertain and even time‐varying due to some unexpected phenomena, such as severe accidents and temporal lane closures. Thus, the RM approach should be able to enhance traffic flow stability by effectively handling the time‐delay effect and fluctuations in traffic flow rate caused by uncertain bottlenecks. This study proposed a novel approach called deep reinforcement learning with curriculum learning (DRLCL) to improve ramp metering efficacy under uncertain bottleneck conditions. The curriculum learning method transfers an optimal control policy from a simple on‐ramp bottleneck case to more challenging bottleneck tasks, while DRLCL agents explore and learn from the tasks gradually. Four RM control tasks were developed in the modified cell transmission model, including typical on‐ramp bottleneck, fixed downstream bottleneck, random‐location bottleneck, and multiple bottlenecks. With curriculum learning, the entire training process was reduced by 45.1% to 64.5%, while maintaining a similar maximum reward level compared to DRL‐based RM control with full learning from scratch. Specifically, the results also demonstrated that the proposed DRLCL‐based RM outperformed the feedback‐based RM due to its stronger predictive ability, faster response, and higher action precision.

Fuzzy Self‐adaptive Proportional–integral–derivative Control Strategy for Ramp Metering at Distance Downstream Bottlenecks

Local Ramp Metering in the Case of Distant Downstream Bottlenecks

Local Ramp Metering with Distant Downstream Bottlenecks: A Comparative Study.

Freeway Traffic Control in Presence of Capacity Drop.

Preventing Freeway Capacity Drop Via MTFC and Ramp Metering.

Local Ramp Metering in Random-Location Bottlenecks Downstream of Metered On-Ramp

An Adaptive Control Scheme for Local and Coordinated Ramp Metering

When to Control the Ramps on Freeway Corridors? A Novel Stability-and-MFD-Based Approach

Integrated Control of Internal Boundary and Ramp Inflows for Lane-free Traffic of Automated Vehicles on Freeways.

Distant Downstream Bottlenecks in Local Ramp Metering: Comparison of Fuzzy Self-Adaptive PID Controller and PI-ALINEA

Capacity Drop at Freeway Ramp Merges with Its Replication in Macroscopic and Microscopic Traffic Simulations: A Tutorial Report

Local Ramp Metering in the Presence of A Distant Downstream Bottleneck: Theoretical Analysis and Simulation Study

Gating Control for a Single Bottleneck Link Based on Traffic Load Equilibrium

An Extended Model Predictive Control Approach To Coordinated Ramp Metering

Enhancing reinforcement learning‐based ramp metering performance at freeway uncertain bottlenecks using curriculum learning

On-ramp metering control strategy of urban expressway

Arterial-Friendly Local Ramp Metering Control Strategy

Adaptive Ramp Metering Control for Urban Freeway Using Large-Scale Data

Integrated control of highway traffic flow

An extension of local ramp metering strategy of expressway considering flow maximizing