Abstract:This study presents a robust deep reinforcement learning (RL) approach for real-time, network-wide holding control with transfer synchronization, considering stochastic passenger demand and vehicle running time during daily operations. The problem is formulated within a multi-agent RL framework where each active trip is considered as an agent, which not only interacts with the environment but also with other agents in the considered transit network. A specific learning procedure is developed to learn robust policies by introducing $maxmin$ optimization into the learning objective. The agents are trained via deep deterministic policy gradient algorithm (DDPG) using an extended actor-critic framework with a joint action approximator. The effectiveness of the proposed approach is evaluated in a simulator, which is calibrated using data collected from a transit network in Twin Cities Minnesota, USA. The learned policy is compared with no control, rule-based control and the rolling horizon optimization control (RHOC). Computational results suggest that RL approach can significantly reduce the online computation time by about 50% compared with RHOC. In terms of policy performance, under deterministic scenario, the average waiting time of RL approach is 1.3% higher than the theoretical lower bound of average waiting time; under stochastic scenarios, RL approach could reduce as much as 18% average waiting time than RHOC, and the performance relative to RHOC improves when the level of system uncertainty increases. Evaluation under disrupted environment also suggests that the proposed RL method is more robust against short term uncertainties. The promising results in terms of both online computational efficiency and solution effectiveness suggest that the proposed RL method is a valid candidate for real-time transit control when the dynamics cannot be modeled perfectly with system uncertainties, as is the case for the network-wide transfer synchronization problem.

Robust Dynamic Bus Control: a Distributional Multi-Agent Reinforcement Learning Approach

Multi-objective multi-agent deep reinforcement learning to reduce bus bunching for multiline services with a shared corridor

Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning

Hybrid Multi-Agent Reinforcement Learning for Electric Vehicle Resilience Control Towards a Low-Carbon Transition

A Robust Integrated Multi-Strategy Bus Control System via Deep Reinforcement Learning

Real-Time Holding Control for Transfer Synchronization Via Robust Multiagent Reinforcement Learning

Robust dynamic real-time control strategies for high-frequency bus service: a multi-agent reinforcement learning framework

Mitigating Bus Bunching via Hierarchical Multi-Agent Reinforcement Learning

Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

Large Language Model-Enhanced Reinforcement Learning for Generic Bus Holding Control Strategies

Integrating Conventional Headway Control with Reinforcement Learning to Avoid Bus Bunching

Enhancing urban metro system resilience under disruptive events through multi-agent reinforcement learning

Dynamic Bus Holding Control Using Spatial-Temporal Data - A Deep Reinforcement Learning Approach

Large-Scale Traffic Signal Control Using a Novel Multiagent Reinforcement Learning

Vehicle Dispatching and Routing of On-Demand Intercity Ride-Pooling Services: A Multi-Agent Hierarchical Reinforcement Learning Approach

Multi-agent Deep Reinforcement Learning for Resilience-Driven Routing and Scheduling of Mobile Energy Storage Systems

Decentralized Deep Reinforcement Learning for Network Level Traffic Signal Control

Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

Minimizing passenger waiting time in the multi-route bus fleet allocation problem through distributionally robust optimization and reinforcement learning

Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control