Abstract:With the construction of high-speed railway networks in China, the traffic density is constantly increasing, and the complexity and difficulty of train rescheduling sharply increase when trains are delayed due to emergencies. How to dynamically adjust the train group operation to reduce the delay and improve the punctuality rate is the core of operation adjustments. In this paper, a model-free reinforcement learning method is proposed for the operation adjustment of high-speed trains in emergency situations. Firstly, the operation adjustment of multiple trains in multiple stations and blocks is modeled as a sequential multi-stage decision-making process of constrained resource occupation and allocation, and a dynamic spatio-temporal topological matrix is proposed to model the stations and blocks. In view of the strong spatio-temporal correlation of high-speed railway trains, a reinforcement learning state space, action space and reward function containing spatio-temporal distribution information such as vehicle positions and network resources are proposed for the first time, and an effective reward feedback mechanism is constructed. Then, aiming at the difficulty of huge search space in high-speed railway operation systems, this paper proposes an adaptive generation method of heuristic action subspace. This method uses some explicit static constraints to construct heuristic rules to reduce the search space, which can effectively reduce the trial-and-error times of model-free reinforcement learning, improve the efficiency of solution, and retain the advantages of model-free generality. Finally, the simulation analysis of the case based on the actual data of Beijing-Guangzhou high-speed railway shows that the proposed algorithm can converge well and significantly reduce the delay propagation within the train group under multiple delays caused by the high wind speed limit in different space and time ranges. Compared with MILP, ACO, and FCFS algorithms, the proposed method can reduce the average delay time of the train group by 2%–20%.

A Policy-based Reinforcement Learning Approach for High-speed Railway Timetable Rescheduling

Rescheduling of High-Speed Trains: a Reinforcement Learning Approach

Reinforcement Learning for Online Dispatching Policy in Real-Time Train Timetable Rescheduling

A Deep Reinforcement Learning Approach to High-speed Train Timetable Rescheduling under Disturbances.

High-speed Train Timetabling Based on Reinforcement Learning.

A Deep Reinforcement Learning Approach for the Traffic Management of High-Speed Railways.

A Driving Strategy Based Integrated Rescheduling Model for High-Speed Railway by Using the Parallel Intelligent Method.

Deep Reinforcement Learning for Integration of Train Trajectory Optimization and Timetable Rescheduling Under Disturbances

A Reinforcement Learning Framework for Train Rescheduling.

Integrated train timetabling and rolling stock rescheduling for a disturbed metro system: A hybrid deep reinforcement learning and adaptive large neighborhood search approach

A Timetable Rescheduling Approach for Railway based on Monte Carlo Tree Search

Traffic Modeling and Rescheduling for High-speed Train Based on Block Sections

Multi-stage timetable rescheduling for high-speed railways: a dynamic programming approach with adaptive state generation

A Real-Time Rescheduling Approach Using Loop Iteration for High-Speed Railway Traffic

A dynamic rescheduling and speed management approach for high-speed trains with uncertain time-delay

Optimization Based High-Speed Railway Train Rescheduling with Speed Restriction

A Multistage Decision Optimization Approach for Train Timetable Rescheduling Under Uncertain Disruptions in a High-Speed Railway Network

A Mobile Data-Driven Hierarchical Deep Reinforcement Learning Approach for Real-time Demand-Responsive Railway Rescheduling and Station Overcrowding Mitigation

An Optimization Method of High-Speed Railway Rescheduling to Meet Unexpected Large Passenger Flow

Integrated Rescheduling of Train Timetables and Rolling Stock Circulation for Metro Line Disturbance Management: a Q-learning-based Approach

State-Space Modeling and Feedback Control for Real-Time Automatic Train Timetable Rescheduling of Intercity HSRs