Abstract:The large-scale integration of distributed energy resources into the energy industry enables the fast transition to a decarbonized future but raises some potential challenges of insecure and unreliable operations. Multi-energy Microgrids (MEMGs), as localized small multi-energy systems, can effectively integrate a variety of energy components with multiple energy sectors, which have been recently recognized as a valid solution to improve the operational security and reliability. As a result, a massive amount of research has been conducted to investigate MEMG energy management problems, including both model-based optimization and model-free learning approaches. Compared to optimization approaches, reinforcement learning is being widely deployed in MEMG energy management problems owing to its ability to handle highly dynamic and stochastic processes without knowing any system knowledge. However, it is still difficult for conventional model-free reinforcement learning methods to capture the physical constraints of the MEMG model, which may therefore destroy its secure operation. To address this research challenge, this paper proposes a novel safe reinforcement learning method by learning a dynamic security assessment rule to abstract a physical-informed safety layer on top of the conventional model-free reinforcement learning energy management policy, which can respect all the physical constraints through mathematically solving an action correction formulation. In this setting, the secure energy management of the MEMG can be guaranteed for both training and test procedures. Extensive case studies based on two integrated systems (i.e., a small 6-bus power and 7-node gas network, and a large 33-bus power and 20-node gas network) are carried out to verify the superior performance of the proposed physical-informed reinforcement learning method in achieving a cost-effective MEMG energy management performance while respecting all the physical constraints, compared to conventional reinforcement learning and optimization approaches.

Reinforcement learning-based event-triggered secondary control of DC microgrids

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

Distributed Secondary Control for DC Microgrid with Event-Triggered Signal Transmissions

Distributed Event-Triggered Secondary Control for Islanded Microgrids With Proper Trigger Condition Checking Period

Optimal Distributed Coordinated Reinforcement Learning for Secondary Voltage Control in Time-Delayed Microgrid

Reinforcement Learning Based Integrated Control to Improve the Efficiency of DC Microgrids

Resilient Reinforcement Learning for Voltage Control in an Islanded DC Microgrid Integrating Data-Driven Piezoelectric

Design and Implementation of an Energy Management System with Event-Triggered Distributed Secondary Control in DC Microgrids

Reinforcement learning robust nonlinear control of a microgrid with hybrid energy storage systems

Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Reinforcement Learning$\unicode{x2013}$Based Transient Response Shaping for Microgrids

Data-Efficient Deep Reinforcement Learning-Based Optimal Generation Control in DC Microgrids

Hybrid Control of the DC Microgrid Using Deep Neural Networks and Global Terminal Sliding Mode Control with the Exponential Reaching Law

Multi-Agent Reinforcement Learning for Decentralized Resilient Secondary Control of Energy Storage Systems Against DoS Attacks

Distributed Event-Triggered Control of DC Microgrids With Input Saturation and Time Delay Constraints

Distributed Learning-Based Secondary Control for Islanded DC Microgrids: A High-Order Fully Actuated System Approach

Learning-Based Model Predictive Control of DC-DC Buck Converters in DC Microgrids: A Multi-Agent Deep Reinforcement Learning Approach

Reinforcement Learning for Energy Management of an Islanded Microgrid: Analysis of State, State Space and Reinforcement Function

Real-time Energy Management of Microgrid Using Reinforcement Learning

Microgrid Energy Management Using Improved Reinforcement Learning with Quadratic Programming

Distributed secondary control for DC microgrid under the adaptive event-triggered protocol