Abstract:In metro system, the fault of traction power supply system may cause the power supply shortage around the failure substation. In this case, the dispatching measure should be immediately taken to reduce the impacts of disruption on the train operation. To deal with this real-time traffic management problem, a cooperative control approach is proposed in this paper. In this approach, the time to apply tractive force and the level of force are simultaneously adjusted for all the operated trains, to maximize the maintained line capacity when considering the power supply capacity. Compared with the existing train timetable rescheduling approach, cooperative control is more flexible to get a better train regulation solution. To solve the challenges for developing the cooperative control model (i.e., undetermined number and dynamically changing of controlled objects), an imaginary section method is newly developed to transform the original problem into an equivalent cooperative control problem with fixed controlled objects. Then, the mathematical models for the transformed problem are constructed by using the space–time–speed network methodology. According to the formulated model, a Decentralized-Markov Decision Process (Dec-MDP) framework is designed as the basis of the applied algorithm. Next, a Collaboration Mechanism Based-Independent Deep Q-Network (CMB-IDQN) algorithm is proposed to solve the cooperative control problem. Compared with classical IDQN algorithm, a credit assignment method based on the collaboration mechanism among trains is novelly considered in the designed multi-agent reinforcement learning algorithm. Finally, the effectiveness of the proposed cooperative control approach is verified by two case studies. When solving the cooperative control problem, the performance by using CMB-IDQN algorithm can be increased by up to 13.0% and 16.8% compared with other two classical reinforcement learning algorithms (i.e., DQN and IDQN), respectively. Compared with two train timetable rescheduling measures during the power supply shortage, the cooperative control approach can improve the solution quality by more than 180.4% and 17.4%, respectively.

Research of security system for amusement facilities based on improved multi-agent cooperative reinforcement learning method

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Remote Security Monitoring and Warning-Forecasting System for Large-Scale Amusement Ride Based on Multi-Agent

Cooperative Flocking And Learning In Multi-Robot Systems For Predator Avoidance

MAS4AMR: A Self Organized Multi Agent System Designed for Auto Mobile Robot

Safe Exploration in Wireless Security: A Safe Reinforcement Learning Algorithm With Hierarchical Structure

Improved Multi-Agent Reinforcement Learning for Path Planning-Based Crowd Simulation

Research on intelligent algorithm for alerting vehicle impact based on multi-agent deep reinforcement learning

Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replay

Attention-Based Fault-Tolerant Approach for Multi-Agent Reinforcement Learning Systems

A Multi-Agents Deep Reinforcement Learning Autonomous Security Management Approach for Internet of Things

Learning Adaptive Safety for Multi-Agent Systems

Prior Knowledge-Augmented Broad Reinforcement Learning Framework for Fault Diagnosis of Heterogeneous Multiagent Systems

Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

Substation intelligent security management and control practice based on artificial intelligence and big data analysis

Study on Learning Algorithm of Transfer Reinforcement for Multi-Agent Formation Control

A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehicles

PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety

The Design and Realization of Multi-agent Obstacle Avoidance based on Reinforcement Learning

Security and Safety-Critical Learning-Based Collaborative Control for Multiagent Systems

Cooperative train control during the power supply shortage in metro system: A multi-agent reinforcement learning approach