Abstract:Extreme events are greatly impacting the normal operations of microgrids, which can lead to severe outages and affect the continuous supply of energy to customers, incurring substantial restoration costs. Repair crews (RCs) are regarded as crucial resources to provide system resilience owing to their mobility and flexibility characteristics in handling both transportation and energy systems. Nevertheless, effectively coordinating the dispatch of RCs towards system resilience is a complex decision-making problem, especially in the context of a multi-energy microgrid (MEMG) with enormous dynamics and uncertainties. To this end, this paper formulates the dispatch problem of RCs in a coupled transportation and power-gas network as a decentralized partially observable Markov decision process (Dec-POMDP). To solve this Dec-POMDP, a hierarchical multi-agent reinforcement learning (MARL) algorithm is proposed by featuring a two-level framework, where the high-level action is used for switching decision-making between transportation and power-gas networks, and the lower-level action constructed via the multi-agent proximal policy optimization (MAPPO) algorithm is used to compute the routing and repairing decisions of RCs in the transportation and power-gas networks, respectively. The proposed algorithm also introduces an abstracted critic network by integrating the load restoration status, which captures the system dynamics and stabilizes the training performance with privacy protection. Extensive case studies are evaluated on a coupled 6-bus power and 6-bus gas network integrated with a 9-node 12-edge transportation network. The proposed algorithm outperforms the conventional MARL algorithms in terms of policy quality, learning stability, and computational performance. Furthermore, the dispatch strategies of RCs are analyzed and their corresponding benefits for load restoration are also evaluated. Finally, the scalability of the proposed method is also investigated for a larger 33-bus power and 15-bus gas network integrated with an 18-node 27-edge transportation network.

Coordinating Multi-Energy Microgrids for Integrated Energy System Resilience: A Multi-Task Learning Approach

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

Synergistic Resilient Strategy of Integrated Electric and Gas System: an ADMM-based Dispatch Framework Considering Joint Working Group

Research on Real-Time Coordinated Optimization Scheduling Control Strategy with Supply-Side Flexibility in Multi-Microgrid Energy Systems

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

Distributed Optimal Tie-Line Power Flow Control for Multiple Interconnected AC Microgrids

Resilience-Oriented Coordination of Networked Microgrids: a Shapley Q-Value Learning Approach

Towards Microgrid Resilience Enhancement Via Mobile Power Sources and Repair Crews: A Multi-Agent Reinforcement Learning Approach

Hierarchical multi-agent reinforcement learning for repair crews dispatch control towards multi-energy microgrid resilience

Distributed Optimal Scheduling of Multi-Microgrid System with Hydrogen Based on Model Predictive Control

Smart Coordinated Multi-Energy Intra-Scheduling Inter-Sharing and Cost-Saving Redistribution for Multiple Microgrids

Multi-agent Deep Reinforcement Learning for Resilience-Driven Routing and Scheduling of Mobile Energy Storage Systems

Deep Reinforcement Learning Based Optimal Energy Management of Multi-energy Microgrids with Uncertainties

Multi-Agent Learning-Based Nearly Non-Iterative Stochastic Dynamic Transactive Energy Control of Networked Microgrids

Multi-Stage Real-Time Operation of a Multi-Energy Microgrid With Electrical and Thermal Energy Storage Assets: A Data-Driven MPC-ADP Approach

A Resilience Enhancement Strategy for Networked Microgrids Incorporating Electricity and Transport and Utilizing a Stochastic Hierarchical Control Approach

Multi-stage optimal energy management of multi-energy microgrid in deregulated electricity markets

A Stochastic Distributed Control Approach for Load Restoration of Networked Microgrids with Mobile Energy Storage Systems

Operation and Coordinated Energy Management in Multi-Microgrids for Improved and Resilient Distributed Energy Resource Integration in Power Systems

Collaborative Optimization of Multi-microgrids System with Shared Energy Storage Based on Multi-agent Stochastic Game and Reinforcement Learning

Cyber-physical-social system scheduling for multi-energy microgrids with distribution network coordination