Abstract:The large-scale integration of distributed energy resources into the energy industry enables the fast transition to a decarbonized future but raises some potential challenges of insecure and unreliable operations. Multi-energy Microgrids (MEMGs), as localized small multi-energy systems, can effectively integrate a variety of energy components with multiple energy sectors, which have been recently recognized as a valid solution to improve the operational security and reliability. As a result, a massive amount of research has been conducted to investigate MEMG energy management problems, including both model-based optimization and model-free learning approaches. Compared to optimization approaches, reinforcement learning is being widely deployed in MEMG energy management problems owing to its ability to handle highly dynamic and stochastic processes without knowing any system knowledge. However, it is still difficult for conventional model-free reinforcement learning methods to capture the physical constraints of the MEMG model, which may therefore destroy its secure operation. To address this research challenge, this paper proposes a novel safe reinforcement learning method by learning a dynamic security assessment rule to abstract a physical-informed safety layer on top of the conventional model-free reinforcement learning energy management policy, which can respect all the physical constraints through mathematically solving an action correction formulation. In this setting, the secure energy management of the MEMG can be guaranteed for both training and test procedures. Extensive case studies based on two integrated systems (i.e., a small 6-bus power and 7-node gas network, and a large 33-bus power and 20-node gas network) are carried out to verify the superior performance of the proposed physical-informed reinforcement learning method in achieving a cost-effective MEMG energy management performance while respecting all the physical constraints, compared to conventional reinforcement learning and optimization approaches.

Data-Driven Security and Stability Rule in High Renewable Penetrated Power System Operation

Data-driven approaches for power system operation and planning under high renewable energy penetration

Analysis of Key Factors Influencing Frequency Security in Power System Operation Mode

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

A Confidence-Aware Machine Learning Framework for Dynamic Security Assessment

Cybersecurity Analysis of Data-Driven Power System Stability Assessment

Hybrid Model and Data Driven Concepts for Power System Security Feature Selection and Knowledge Discovery:Key Technologies and Engineering Application

Understanding the Safety Requirements for Learning-based Power Systems Operations

Machine Learning-Based Method for Power System Operation Rules Generation

Sparse Oblique Decision Tree for Power System Security Rules Extraction and Embedding

Dynamic Reactive Power Configuration of High Penetration Renewable Energy Grid Based on Transient Stability Probability Assessment

On Machine Learning-Based Techniques for Future Sustainable and Resilient Energy Systems

A Deep Learning-Based Feature Extraction Framework for System Security Assessment

Power System Stability Issues, Classifications And Research Prospects In The Context Of High-Penetration Of Renewables And Power Electronics

A Deep Spatial-Temporal Data-Driven Approach Considering Microclimates for Power System Security Assessment

Automatic Learning of Fine Operating Rules for Online Power System Security Control

Data-driven Research Method For Power System Stability Detection

Is Machine Learning in Power Systems Vulnerable?

Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control

Research on Power System Transient Stability Assessment Based on Deep Learning of Big Data Technique