Abstract:In metro system, the fault of traction power supply system may cause the power supply shortage around the failure substation. In this case, the dispatching measure should be immediately taken to reduce the impacts of disruption on the train operation. To deal with this real-time traffic management problem, a cooperative control approach is proposed in this paper. In this approach, the time to apply tractive force and the level of force are simultaneously adjusted for all the operated trains, to maximize the maintained line capacity when considering the power supply capacity. Compared with the existing train timetable rescheduling approach, cooperative control is more flexible to get a better train regulation solution. To solve the challenges for developing the cooperative control model (i.e., undetermined number and dynamically changing of controlled objects), an imaginary section method is newly developed to transform the original problem into an equivalent cooperative control problem with fixed controlled objects. Then, the mathematical models for the transformed problem are constructed by using the space–time–speed network methodology. According to the formulated model, a Decentralized-Markov Decision Process (Dec-MDP) framework is designed as the basis of the applied algorithm. Next, a Collaboration Mechanism Based-Independent Deep Q-Network (CMB-IDQN) algorithm is proposed to solve the cooperative control problem. Compared with classical IDQN algorithm, a credit assignment method based on the collaboration mechanism among trains is novelly considered in the designed multi-agent reinforcement learning algorithm. Finally, the effectiveness of the proposed cooperative control approach is verified by two case studies. When solving the cooperative control problem, the performance by using CMB-IDQN algorithm can be increased by up to 13.0% and 16.8% compared with other two classical reinforcement learning algorithms (i.e., DQN and IDQN), respectively. Compared with two train timetable rescheduling measures during the power supply shortage, the cooperative control approach can improve the solution quality by more than 180.4% and 17.4%, respectively.

Decentralized Cooperative Control of Multiple Energy Storage Systems in Urban Railway Based on Multiagent Deep Reinforcement Learning

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Energy-efficient Operation by Cooperative Control among Trains: A Multi-Agent Reinforcement Learning Approach

Cooperative train control during the power supply shortage in metro system: A multi-agent reinforcement learning approach

Coordinated Control of the Onboard and Wayside Energy Storage System of an Urban Rail Train Based on Rule Mining

Deep-Reinforcement-Learning-Based Energy Management Strategy for Supercapacitor Energy Storage Systems in Urban Rail Transit

A Multi-Agent Reinforcement Learning-Based Method for Multiple Electric Vehicles Charging Scheduling

Energy-Saving Train Operation Synergy Based on Multi-Agent Deep Reinforcement Learning on Spark Cloud

Scene-segmentation Based Control Strategy of Energy Storage System for Urban Railway

Active Collaborative Recovery Method of Regenerative Braking Energy by Multiple Energy Storage Systems in Urban Railway

Power Allocation Strategy for Urban Rail HESS Based on Deep Reinforcement Learning Sequential Decision Optimization

Control Method of Urban Rail Energy Storage System Based on Real-time Correction of Train Timetable and Train Power Following

A Cooperative Charging Control Strategy for Electric Vehicles Based on Multiagent Deep Reinforcement Learning

Cooperative Control of Metro Trains to Minimize Net Energy Consumption

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Multistep Multiagent Reinforcement Learning for Optimal Energy Schedule Strategy of Charging Stations in Smart Grid.

A Multi-Agent Deep Reinforcement Learning Approach Enabled Distributed Energy Management Schedule for the Coordinate Control of Multi-Energy Hub with Gas, Electricity, and Freshwater

Real-Time Energy-Saving Optimization for Multiple Trains Based on Multiagent Cooperative Control

Decentralized multi-objective cloud energy storage operation control with deep reinforcement learning

Multi Time Scale Management and Coordination Strategy for Stationary Super Capacitor Energy Storage in Urban Rail Transit Power Supply System

Edge Computing Based Coordinated Energy-saving Train Regulation with Multi-agent Learning