Abstract:The large-scale integration of distributed energy resources into the energy industry enables the fast transition to a decarbonized future but raises some potential challenges of insecure and unreliable operations. Multi-energy Microgrids (MEMGs), as localized small multi-energy systems, can effectively integrate a variety of energy components with multiple energy sectors, which have been recently recognized as a valid solution to improve the operational security and reliability. As a result, a massive amount of research has been conducted to investigate MEMG energy management problems, including both model-based optimization and model-free learning approaches. Compared to optimization approaches, reinforcement learning is being widely deployed in MEMG energy management problems owing to its ability to handle highly dynamic and stochastic processes without knowing any system knowledge. However, it is still difficult for conventional model-free reinforcement learning methods to capture the physical constraints of the MEMG model, which may therefore destroy its secure operation. To address this research challenge, this paper proposes a novel safe reinforcement learning method by learning a dynamic security assessment rule to abstract a physical-informed safety layer on top of the conventional model-free reinforcement learning energy management policy, which can respect all the physical constraints through mathematically solving an action correction formulation. In this setting, the secure energy management of the MEMG can be guaranteed for both training and test procedures. Extensive case studies based on two integrated systems (i.e., a small 6-bus power and 7-node gas network, and a large 33-bus power and 20-node gas network) are carried out to verify the superior performance of the proposed physical-informed reinforcement learning method in achieving a cost-effective MEMG energy management performance while respecting all the physical constraints, compared to conventional reinforcement learning and optimization approaches.

Real-world validation of safe reinforcement learning, model predictive control and decision tree-based home energy management systems

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

TreeC: a method to generate interpretable energy management systems using a metaheuristic algorithm

Explainable Reinforcement Learning-based Home Energy Management Systems using Differentiable Decision Trees

Energy Management Based on Safe Multi-Agent Reinforcement Learning for Smart Buildings in Distribution Networks

Safe Reinforcement Learning for Real-Time Automatic Control in a Smart Energy-Hub

Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates

Evaluating Reinforcement Learning Algorithms in Residential Energy Saving and Comfort Management

Deep Reinforcement Learning for Real-Time Energy Management in Smart Home

A Human-Machine Reinforcement Learning Method for Cooperative Energy Management.

A Home Energy Management Approach Using Decoupling Value and Policy in Reinforcement Learning

Optimal Management of Grid-Interactive Efficient Buildings via Safe Reinforcement Learning

Real-Time Energy Management in Smart Homes Through Deep Reinforcement Learning

Comparative Analysis of Reinforcement Learning Approaches for Multi-Objective Optimization in Residential Hybrid Energy Systems

Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control

An adaptive safety layer with hard constraints for safe reinforcement learning in multi-energy management systems

Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control

Model-Free Real-Time Autonomous Energy Management for a Residential Multi-Carrier Energy System: A Deep Reinforcement Learning Approach

Deep reinforcement learning based home energy management system with devices operational dependencies

Distill2Explain: Differentiable decision trees for explainable reinforcement learning in energy application controllers