Abstract:In metro system, the fault of traction power supply system may cause the power supply shortage around the failure substation. In this case, the dispatching measure should be immediately taken to reduce the impacts of disruption on the train operation. To deal with this real-time traffic management problem, a cooperative control approach is proposed in this paper. In this approach, the time to apply tractive force and the level of force are simultaneously adjusted for all the operated trains, to maximize the maintained line capacity when considering the power supply capacity. Compared with the existing train timetable rescheduling approach, cooperative control is more flexible to get a better train regulation solution. To solve the challenges for developing the cooperative control model (i.e., undetermined number and dynamically changing of controlled objects), an imaginary section method is newly developed to transform the original problem into an equivalent cooperative control problem with fixed controlled objects. Then, the mathematical models for the transformed problem are constructed by using the space–time–speed network methodology. According to the formulated model, a Decentralized-Markov Decision Process (Dec-MDP) framework is designed as the basis of the applied algorithm. Next, a Collaboration Mechanism Based-Independent Deep Q-Network (CMB-IDQN) algorithm is proposed to solve the cooperative control problem. Compared with classical IDQN algorithm, a credit assignment method based on the collaboration mechanism among trains is novelly considered in the designed multi-agent reinforcement learning algorithm. Finally, the effectiveness of the proposed cooperative control approach is verified by two case studies. When solving the cooperative control problem, the performance by using CMB-IDQN algorithm can be increased by up to 13.0% and 16.8% compared with other two classical reinforcement learning algorithms (i.e., DQN and IDQN), respectively. Compared with two train timetable rescheduling measures during the power supply shortage, the cooperative control approach can improve the solution quality by more than 180.4% and 17.4%, respectively.

Deterministic reinforcement learning for optimized formation control of virtually-coupled trains via performance index monitor

Training for More Robust and Practical Adaptive Signal Control Models

Iterative Learning Tracking Control of High-Speed Trains with Nonlinearly Parameterized Uncertainties and Multiple Time-Varying Delays

Switching Learning-Based Cooperative Control with Its Application to Connected Automated Vehicles

Reinforcement Learning-Based Unknown Reference Tracking Control of HMASs with Nonidentical Communication Delays

Leader-follower Formation Control for a Multi-missile System Via Deep Reinforcement Learning

Train Trajectory Optimization with High-Risk State Space Boundaries: A Safe Reinforcement Learning Approach

Adaptive optimal formation control for unmanned surface vehicles with guaranteed performance using actor‐critic learning architecture

Deep Deterministic Policy Gradient Virtual Coupling control for the coordination and manoeuvring of heterogeneous uncertain nonlinear High-Speed Trains

Synthesis of dynamic modelling framework and optimal control strategy for virtually coupled train sets with guaranteed safety

Multi-Agent System Based Cooperative Control for Speed Convergence of Virtually Coupled Train Formation

Adaptive Fixed-time Optimal Formation Control for Uncertain Nonlinear Multiagent Systems Using Reinforcement Learning

Cooperative deterministic learning and formation control for underactuated USVs with prescribed performance

Event-Triggered Optimal Formation Tracking Control Using Reinforcement Learning for Large-Scale UAV Systems

Decentralized Multi-Robot Formation Control Using Reinforcement Learning

Distributed output formation tracking control of heterogeneous multi-agent systems using reinforcement learning

Reinforcement learning-based close formation control for underactuated surface vehicle with prescribed performance and time-varying state constraints

Event-based adaptive formation and tracking control with predetermined performance for nonlinear multi-agent systems

Heterogeneous formation control of multiple rotorcrafts with unknown dynamics by reinforcement learning

An Intelligent Train Operation Method Based on Event-Driven Deep Reinforcement Learning

Cooperative train control during the power supply shortage in metro system: A multi-agent reinforcement learning approach