Abstract:The large-scale integration of distributed energy resources into the energy industry enables the fast transition to a decarbonized future but raises some potential challenges of insecure and unreliable operations. Multi-energy Microgrids (MEMGs), as localized small multi-energy systems, can effectively integrate a variety of energy components with multiple energy sectors, which have been recently recognized as a valid solution to improve the operational security and reliability. As a result, a massive amount of research has been conducted to investigate MEMG energy management problems, including both model-based optimization and model-free learning approaches. Compared to optimization approaches, reinforcement learning is being widely deployed in MEMG energy management problems owing to its ability to handle highly dynamic and stochastic processes without knowing any system knowledge. However, it is still difficult for conventional model-free reinforcement learning methods to capture the physical constraints of the MEMG model, which may therefore destroy its secure operation. To address this research challenge, this paper proposes a novel safe reinforcement learning method by learning a dynamic security assessment rule to abstract a physical-informed safety layer on top of the conventional model-free reinforcement learning energy management policy, which can respect all the physical constraints through mathematically solving an action correction formulation. In this setting, the secure energy management of the MEMG can be guaranteed for both training and test procedures. Extensive case studies based on two integrated systems (i.e., a small 6-bus power and 7-node gas network, and a large 33-bus power and 20-node gas network) are carried out to verify the superior performance of the proposed physical-informed reinforcement learning method in achieving a cost-effective MEMG energy management performance while respecting all the physical constraints, compared to conventional reinforcement learning and optimization approaches.

Scalable and Privacy-Preserving Distributed Energy Management for Multimicrogrid

Distributed Energy Management for the Multi-Microgrid System Based on ADMM

Distributed Real-Time Energy Management of Multi-microgrid for Incomplete Information Scenarios

A Secure Distributed Transactive Energy Management Scheme for Multiple Interconnected Microgrids Considering Misbehaviors

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

Privacy-Preserving Decentralized Energy Management for Networked Microgrids via Objective-Based ADMM

Distributed Energy Management of the Hybrid AC/DC Microgrid with High Penetration of Distributed Energy Resources Based on ADMM

Distributed Optimal Energy Management in Microgrids

Multi-energy Microgrid Robust Energy Management with a Novel Decision-Making Strategy

A Stochastic Spatiotemporal Decomposition Decision-Making Approach for Real-Time Dynamic Energy Management of Multi-Microgrids

A Fully Distributed Privacy-Preserving Energy Management System for Networked Microgrid Cluster Based on Homomorphic Encryption

Data-Driven Risk-Adjusted Robust Energy Management for Microgrids Integrating Demand Response Aggregator and Renewable Energies

Robust Energy Management for Microgrids With High-Penetration Renewables

Managing Energy Storage in Microgrids: A Multistage Stochastic Programming Approach

A consensus-based privacy-preserving energy management strategy for microgrids with event-triggered scheme

Multi-Layer Energy Management and Strategy Learning for Microgrids: A Proximal Policy Optimization Approach

A Multi-Agent Deep-Reinforcement-Learning-Based Strategy for Safe Distributed Energy Resource Scheduling in Energy Hubs

A Novel Real-Time Energy Management Strategy for Grid-Supporting Microgrid: Enabling Flexible Trading Power

A Data-Driven Energy Management Strategy Based on Deep Reinforcement Learning for Microgrid Systems

Federated Multiagent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multimicrogrid Energy Management

Day-ahead optimization for smart energy management of multi-microgrid using a stochastic-robust model