Abstract:With the construction of high-speed railway networks in China, the traffic density is constantly increasing, and the complexity and difficulty of train rescheduling sharply increase when trains are delayed due to emergencies. How to dynamically adjust the train group operation to reduce the delay and improve the punctuality rate is the core of operation adjustments. In this paper, a model-free reinforcement learning method is proposed for the operation adjustment of high-speed trains in emergency situations. Firstly, the operation adjustment of multiple trains in multiple stations and blocks is modeled as a sequential multi-stage decision-making process of constrained resource occupation and allocation, and a dynamic spatio-temporal topological matrix is proposed to model the stations and blocks. In view of the strong spatio-temporal correlation of high-speed railway trains, a reinforcement learning state space, action space and reward function containing spatio-temporal distribution information such as vehicle positions and network resources are proposed for the first time, and an effective reward feedback mechanism is constructed. Then, aiming at the difficulty of huge search space in high-speed railway operation systems, this paper proposes an adaptive generation method of heuristic action subspace. This method uses some explicit static constraints to construct heuristic rules to reduce the search space, which can effectively reduce the trial-and-error times of model-free reinforcement learning, improve the efficiency of solution, and retain the advantages of model-free generality. Finally, the simulation analysis of the case based on the actual data of Beijing-Guangzhou high-speed railway shows that the proposed algorithm can converge well and significantly reduce the delay propagation within the train group under multiple delays caused by the high wind speed limit in different space and time ranges. Compared with MILP, ACO, and FCFS algorithms, the proposed method can reduce the average delay time of the train group by 2%–20%.

Reinforcement Learning for Combinatorial Optimization of Train Timetable Rescheduling

A Reinforcement Learning Framework for Train Rescheduling.

A Deep Reinforcement Learning Approach to High-speed Train Timetable Rescheduling under Disturbances.

Reinforcement Learning for Online Dispatching Policy in Real-Time Train Timetable Rescheduling

Reinforcement Learning for Scalable Train Timetable Rescheduling With Graph Representation

Deep Reinforcement Learning for Integration of Train Trajectory Optimization and Timetable Rescheduling Under Disturbances

High-speed Train Timetabling Based on Reinforcement Learning.

Rescheduling of High-Speed Trains: a Reinforcement Learning Approach

A Policy-based Reinforcement Learning Approach for High-speed Railway Timetable Rescheduling

Metro Train Timetable Rescheduling Based on Q-learning Approach

A Deep Reinforcement Learning Approach for the Traffic Management of High-Speed Railways.

Single-track railway scheduling with a novel gridworld model and scalable deep reinforcement learning

Real-Time Optimization of Urban Rail Transit Train Scheduling Via Advantage Actor–Critic Deep Reinforcement Learning

Multi-stage timetable rescheduling for high-speed railways: a dynamic programming approach with adaptive state generation

Deep Reinforcement Learning based dynamic optimization of bus timetable

A Deep Reinforcement Learning Approach for the Energy-Aimed Train Timetable Rescheduling Problem Under Disturbances

Train Trajectory Optimization with High-Risk State Space Boundaries: A Safe Reinforcement Learning Approach

Application of a Deep Deterministic Policy Gradient Algorithm for Energy-Aimed Timetable Rescheduling Problem

DRLS: A Deep Reinforcement Learning Based Scheduler for Time-Triggered Ethernet

Timetable Mapping Model and Dynamic Programming Approach for High-speed Railway Rescheduling