Abstract:This study presents a robust deep reinforcement learning (RL) approach for real-time, network-wide holding control with transfer synchronization, considering stochastic passenger demand and vehicle running time during daily operations. The problem is formulated within a multi-agent RL framework where each active trip is considered as an agent, which not only interacts with the environment but also with other agents in the considered transit network. A specific learning procedure is developed to learn robust policies by introducing $maxmin$ optimization into the learning objective. The agents are trained via deep deterministic policy gradient algorithm (DDPG) using an extended actor-critic framework with a joint action approximator. The effectiveness of the proposed approach is evaluated in a simulator, which is calibrated using data collected from a transit network in Twin Cities Minnesota, USA. The learned policy is compared with no control, rule-based control and the rolling horizon optimization control (RHOC). Computational results suggest that RL approach can significantly reduce the online computation time by about 50% compared with RHOC. In terms of policy performance, under deterministic scenario, the average waiting time of RL approach is 1.3% higher than the theoretical lower bound of average waiting time; under stochastic scenarios, RL approach could reduce as much as 18% average waiting time than RHOC, and the performance relative to RHOC improves when the level of system uncertainty increases. Evaluation under disrupted environment also suggests that the proposed RL method is more robust against short term uncertainties. The promising results in terms of both online computational efficiency and solution effectiveness suggest that the proposed RL method is a valid candidate for real-time transit control when the dynamics cannot be modeled perfectly with system uncertainties, as is the case for the network-wide transfer synchronization problem.

Multi-objective multi-agent deep reinforcement learning to reduce bus bunching for multiline services with a shared corridor

Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Mitigating Bus Bunching via Hierarchical Multi-Agent Reinforcement Learning

Robust Dynamic Bus Control: a Distributional Multi-Agent Reinforcement Learning Approach

A Robust Integrated Multi-Strategy Bus Control System via Deep Reinforcement Learning

Robust dynamic real-time control strategies for high-frequency bus service: a multi-agent reinforcement learning framework

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

Large Language Model-Enhanced Reinforcement Learning for Generic Bus Holding Control Strategies

Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

Integrating Conventional Headway Control with Reinforcement Learning to Avoid Bus Bunching

RL-MSA: a Reinforcement Learning-based Multi-line bus Scheduling Approach

Real-Time Holding Control for Transfer Synchronization Via Robust Multiagent Reinforcement Learning

A Collaborative Optimization Method for Train Scheduling and Passenger Flow Assignment Based on Multi-Agent Reinforcement Learning

Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

Multi-Vehicle Cooperative Decision-Making in Merging Area Based on Deep Multi-Agent Reinforcement Learning

Decentralized Deep Reinforcement Learning for Network Level Traffic Signal Control

Multi-agent deep reinforcement learning based real-time planning approach for responsive customized bus routes

Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic