Abstract:The large-scale integration of distributed energy resources into the energy industry enables the fast transition to a decarbonized future but raises some potential challenges of insecure and unreliable operations. Multi-energy Microgrids (MEMGs), as localized small multi-energy systems, can effectively integrate a variety of energy components with multiple energy sectors, which have been recently recognized as a valid solution to improve the operational security and reliability. As a result, a massive amount of research has been conducted to investigate MEMG energy management problems, including both model-based optimization and model-free learning approaches. Compared to optimization approaches, reinforcement learning is being widely deployed in MEMG energy management problems owing to its ability to handle highly dynamic and stochastic processes without knowing any system knowledge. However, it is still difficult for conventional model-free reinforcement learning methods to capture the physical constraints of the MEMG model, which may therefore destroy its secure operation. To address this research challenge, this paper proposes a novel safe reinforcement learning method by learning a dynamic security assessment rule to abstract a physical-informed safety layer on top of the conventional model-free reinforcement learning energy management policy, which can respect all the physical constraints through mathematically solving an action correction formulation. In this setting, the secure energy management of the MEMG can be guaranteed for both training and test procedures. Extensive case studies based on two integrated systems (i.e., a small 6-bus power and 7-node gas network, and a large 33-bus power and 20-node gas network) are carried out to verify the superior performance of the proposed physical-informed reinforcement learning method in achieving a cost-effective MEMG energy management performance while respecting all the physical constraints, compared to conventional reinforcement learning and optimization approaches.

Reinforcement Learning with Probabilistic Boolean Network Models of Smart Grid Devices

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

Modeling failures in smart grids by a bilinear logistic regression approach

A Novel Model-Based Reinforcement Learning for Online Anomaly Detection in Smart Power Grid

Reinforcement Learning for Resilient Power Grids

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Energy Management Based on Safe Multi-Agent Reinforcement Learning for Smart Buildings in Distribution Networks

Real-time outage management in active distribution networks using reinforcement learning over graphs

Robust Defense Against Extreme Grid Events Using Dual-Policy Reinforcement Learning Agents

Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control

Hierarchical Reinforcement Learning for Power Network Topology Control

Reinforcement Learning for Electricity Network Operation

Graph Reinforcement Learning for Power Grids: A Comprehensive Survey

A Safe Pre-Decision Making Approach for Power System Voltage Stability Enhancement Via Safe Deep Reinforcement Learning

RL for Mitigating Cascading Failures: Targeted Exploration via Sensitivity Factors

Explainable reinforcement learning for distribution network reconfiguration

A Critical Review of Safe Reinforcement Learning Techniques in Smart Grid Applications

Deep Reinforcement Learning for Smart Grid Operations: Algorithms, Applications, and Prospects

Bayesian Reinforcement Learning for Automatic Voltage Control under Cyber-Induced Uncertainty

Safe Reinforcement Learning for Emergency LoadShedding of Power Systems