Abstract:The large-scale integration of distributed energy resources into the energy industry enables the fast transition to a decarbonized future but raises some potential challenges of insecure and unreliable operations. Multi-energy Microgrids (MEMGs), as localized small multi-energy systems, can effectively integrate a variety of energy components with multiple energy sectors, which have been recently recognized as a valid solution to improve the operational security and reliability. As a result, a massive amount of research has been conducted to investigate MEMG energy management problems, including both model-based optimization and model-free learning approaches. Compared to optimization approaches, reinforcement learning is being widely deployed in MEMG energy management problems owing to its ability to handle highly dynamic and stochastic processes without knowing any system knowledge. However, it is still difficult for conventional model-free reinforcement learning methods to capture the physical constraints of the MEMG model, which may therefore destroy its secure operation. To address this research challenge, this paper proposes a novel safe reinforcement learning method by learning a dynamic security assessment rule to abstract a physical-informed safety layer on top of the conventional model-free reinforcement learning energy management policy, which can respect all the physical constraints through mathematically solving an action correction formulation. In this setting, the secure energy management of the MEMG can be guaranteed for both training and test procedures. Extensive case studies based on two integrated systems (i.e., a small 6-bus power and 7-node gas network, and a large 33-bus power and 20-node gas network) are carried out to verify the superior performance of the proposed physical-informed reinforcement learning method in achieving a cost-effective MEMG energy management performance while respecting all the physical constraints, compared to conventional reinforcement learning and optimization approaches.

Multi-Stage Real-Time Operation of a Multi-Energy Microgrid With Electrical and Thermal Energy Storage Assets: A Data-Driven MPC-ADP Approach

Distributed Real-Time Energy Management of Multi-microgrid for Incomplete Information Scenarios

Distributed Energy Management for the Multi-Microgrid System Based on ADMM

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

Model Predictive Control of a Solar Power System with Microturbine and Thermochemical Energy Storage

Multi-stage optimal energy management of multi-energy microgrid in deregulated electricity markets

A Stochastic Spatiotemporal Decomposition Decision-Making Approach for Real-Time Dynamic Energy Management of Multi-Microgrids

Adaptively Constrained Stochastic Model Predictive Control for the Optimal Dispatch of Microgrid

Stochastic Model Predictive Control Based Scheduling Optimization of Multi-Energy System Considering Hybrid CHPs and EVs

Distributed Optimal Scheduling of Multi-Microgrid System with Hydrogen Based on Model Predictive Control

Real-time Operation Optimization of Microgrids with Battery Energy Storage System: A Tube-based Model Predictive Control Approach

MPC-driven optimal scheduling of grid-connected microgrid: Cost and degradation minimization with PEVs integration

Multi-time-scale energy management strategy considering battery operation modes for grid-connected microgrids community

Controlling Microgrids Without External Data: A Benchmark of Stochastic Programming Methods

Distributed neurodynamic optimization for multi-energy management with time-varying external disturbances considering time-varying emission limitations and load demand in multi-microgrid

Managing Energy Storage in Microgrids: A Multistage Stochastic Programming Approach

Robust Energy Management for Multi-Microgrids Based on Distributed Dynamic Tube Model Predictive Control

An Intelligent Control Strategy of Battery Energy Storage System for Microgrid Energy Management under Forecast Uncertainties

A Risk-Averse Adaptive Stochastic Optimization Method for Transactive Energy Management of a Multi-Energy Microgrid

Decentralized bi-level stochastic optimization approach for multi-agent multi-energy networked micro-grids with multi-energy storage technologies

An Energy Management System for the Control of Battery Storage in a Grid-Connected Microgrid Using Mixed Integer Linear Programming