Abstract:This study proposed a novel approach called deep reinforcement learning with curriculum learning (DRLCL) to improve ramp metering efficacy under uncertain bottleneck conditions. The curriculum learning method transfers an optimal control policy from a simple on‐ramp bottleneck case to more challenging bottleneck tasks, while DRLCL agents explore and learn from the tasks gradually. Four RM control tasks were developed in the modified cell transmission model, including typical on‐ramp bottleneck, fixed downstream bottleneck, random‐location bottleneck, and multiple bottlenecks. Most current RM approaches are developed for fixed bottlenecks. However, the number and locations of bottlenecks are usually uncertain and even time‐varying due to some unexpected phenomena, such as severe accidents and temporal lane closures. Thus, the RM approach should be able to enhance traffic flow stability by effectively handling the time‐delay effect and fluctuations in traffic flow rate caused by uncertain bottlenecks. This study proposed a novel approach called deep reinforcement learning with curriculum learning (DRLCL) to improve ramp metering efficacy under uncertain bottleneck conditions. The curriculum learning method transfers an optimal control policy from a simple on‐ramp bottleneck case to more challenging bottleneck tasks, while DRLCL agents explore and learn from the tasks gradually. Four RM control tasks were developed in the modified cell transmission model, including typical on‐ramp bottleneck, fixed downstream bottleneck, random‐location bottleneck, and multiple bottlenecks. With curriculum learning, the entire training process was reduced by 45.1% to 64.5%, while maintaining a similar maximum reward level compared to DRL‐based RM control with full learning from scratch. Specifically, the results also demonstrated that the proposed DRLCL‐based RM outperformed the feedback‐based RM due to its stronger predictive ability, faster response, and higher action precision.

Freeway congestion management on multiple consecutive bottlenecks with RL‐based headway control of autonomous vehicles

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

Deep Reinforcement Learning Based Lane-Level Variable Speed Limit Control

Freeway network traffic management based on distributed reinforcement learning

Integrated Traffic Control for Freeway Recurrent Bottleneck Based on Deep Reinforcement Learning

Operational Characteristics of Mixed-Autonomy Traffic Flow on the Freeway with On- and Off-Ramps and Weaving Sections: an RL-Based Approach

A Freeway Traffic Flow Control Model Based on Distributed Reinforcement Learning

Efficiency and equity based freeway traffic network flow control

Deep Reinforcement Learning to Maximize Arterial Usage during Extreme Congestion

Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

Coordinated Variable Speed Limit Control for Consecutive Bottlenecks on Freeways Using Multiagent Reinforcement Learning

Cooperative Cruising: Reinforcement Learning based Time-Headway Control for Increased Traffic Efficiency

A Novel Framework Combining MPC and Deep Reinforcement Learning With Application to Freeway Traffic Control

A Cyber-Physical System for Freeway Ramp Meter Signal Control Using Deep Reinforcement Learning in a Connected Environment

Enhancing reinforcement learning‐based ramp metering performance at freeway uncertain bottlenecks using curriculum learning

Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning

Traffic Flow Management of Autonomous Vehicles Using Deep Reinforcement Learning and Smart Rerouting

Reinforcement Learning-Based Variable Speed Limit Control Strategy to Reduce Traffic Congestion at Freeway Recurrent Bottlenecks

Multi-Level Objective Control of AVs at a Saturated Signalized Intersection with Multi-Agent Deep Reinforcement Learning Approach

Differential Variable Speed Limits Control for Freeway Recurrent Bottlenecks via Deep Reinforcement learning

A New Solution for Freeway Congestion: Cooperative Speed Limit Control Using Distributed Reinforcement Learning