Abstract:This study proposed a novel approach called deep reinforcement learning with curriculum learning (DRLCL) to improve ramp metering efficacy under uncertain bottleneck conditions. The curriculum learning method transfers an optimal control policy from a simple on‐ramp bottleneck case to more challenging bottleneck tasks, while DRLCL agents explore and learn from the tasks gradually. Four RM control tasks were developed in the modified cell transmission model, including typical on‐ramp bottleneck, fixed downstream bottleneck, random‐location bottleneck, and multiple bottlenecks. Most current RM approaches are developed for fixed bottlenecks. However, the number and locations of bottlenecks are usually uncertain and even time‐varying due to some unexpected phenomena, such as severe accidents and temporal lane closures. Thus, the RM approach should be able to enhance traffic flow stability by effectively handling the time‐delay effect and fluctuations in traffic flow rate caused by uncertain bottlenecks. This study proposed a novel approach called deep reinforcement learning with curriculum learning (DRLCL) to improve ramp metering efficacy under uncertain bottleneck conditions. The curriculum learning method transfers an optimal control policy from a simple on‐ramp bottleneck case to more challenging bottleneck tasks, while DRLCL agents explore and learn from the tasks gradually. Four RM control tasks were developed in the modified cell transmission model, including typical on‐ramp bottleneck, fixed downstream bottleneck, random‐location bottleneck, and multiple bottlenecks. With curriculum learning, the entire training process was reduced by 45.1% to 64.5%, while maintaining a similar maximum reward level compared to DRL‐based RM control with full learning from scratch. Specifically, the results also demonstrated that the proposed DRLCL‐based RM outperformed the feedback‐based RM due to its stronger predictive ability, faster response, and higher action precision.

An Indirect Reinforcement Learning Approach For Ramp Control Under Incident-Induced Congestion

Indirect Reinforcement Learning for Incident-Responsive Ramp Control

Intelligent Ramp Control for Incident Response Using Dyna-Q Architecture

A Highway Entrance Ramp Control Method Based on Deep Reinforcement Learning

Freeway network traffic management based on distributed reinforcement learning

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

A Freeway Traffic Flow Control Model Based on Distributed Reinforcement Learning

A Cyber-Physical System for Freeway Ramp Meter Signal Control Using Deep Reinforcement Learning in a Connected Environment

Reinforcement Learning with Model Predictive Control for Highway Ramp Metering

A Stochastic Adaptive Control Model for Isolated Intersections

A Deep Reinforcement Learning Approach for Isolated Intersection Traffic Signal Control with Long-Short Term Memory Network

Efficiency and equity based freeway traffic network flow control

A Complementary Modularized Ramp Metering Approach Based on Iterative Learning Control and ALINEA

Demonstration-guided Deep Reinforcement Learning for Coordinated Ramp Metering and Perimeter Control in Large Scale Networks

Modified Iterative-Learning-Control-Based Ramp Metering Strategies for Freeway Traffic Control with Iteration-Dependent Factors

Enhancing reinforcement learning‐based ramp metering performance at freeway uncertain bottlenecks using curriculum learning

Deep Reinforcement Learning to Maximize Arterial Usage during Extreme Congestion

A Deep Reinforcement Learning Approach for Ramp Metering Based on Traffic Video Data

Reinforcement Learning for Ramp Control: an Analysis of Learning Parameters