Abstract:The large-scale integration of distributed energy resources into the energy industry enables the fast transition to a decarbonized future but raises some potential challenges of insecure and unreliable operations. Multi-energy Microgrids (MEMGs), as localized small multi-energy systems, can effectively integrate a variety of energy components with multiple energy sectors, which have been recently recognized as a valid solution to improve the operational security and reliability. As a result, a massive amount of research has been conducted to investigate MEMG energy management problems, including both model-based optimization and model-free learning approaches. Compared to optimization approaches, reinforcement learning is being widely deployed in MEMG energy management problems owing to its ability to handle highly dynamic and stochastic processes without knowing any system knowledge. However, it is still difficult for conventional model-free reinforcement learning methods to capture the physical constraints of the MEMG model, which may therefore destroy its secure operation. To address this research challenge, this paper proposes a novel safe reinforcement learning method by learning a dynamic security assessment rule to abstract a physical-informed safety layer on top of the conventional model-free reinforcement learning energy management policy, which can respect all the physical constraints through mathematically solving an action correction formulation. In this setting, the secure energy management of the MEMG can be guaranteed for both training and test procedures. Extensive case studies based on two integrated systems (i.e., a small 6-bus power and 7-node gas network, and a large 33-bus power and 20-node gas network) are carried out to verify the superior performance of the proposed physical-informed reinforcement learning method in achieving a cost-effective MEMG energy management performance while respecting all the physical constraints, compared to conventional reinforcement learning and optimization approaches.

Agent-based Modeling and Reinforcement Learning for Optimizing Energy Systems Operation and Maintenance: The Pathmind Solution

Optimization of the Operation and Maintenance of Renewable Energy Systems by Deep Reinforcement Learning

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

Optimal energy management of energy hub: A reinforcement learning approach

Multi-Agent Reinforcement Learning for Power System Operation and Control

Towards Pareto-optimal energy management in integrated energy systems: A multi-agent and multi-objective deep reinforcement learning approach

Multi-agent hierarchical reinforcement learning for energy management

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Multi-agent deep reinforcement learning-based optimal energy management for grid-connected multiple energy carrier microgrids

Online Operational Decision-making for Integrated Electric-Gas Systems with Safe Reinforcement Learning

Optimizing Maintenance of Energy Supply Systems in City Logistics with Heuristics and Reinforcement Learning

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

Multi-Agent Reinforcement Learning for Power Grid Topology Optimization

DRL-HEMS: Deep Reinforcement Learning Agent for Demand Response in Home Energy Management Systems Considering Customers and Operators Perspectives

A Human-Machine Reinforcement Learning Method for Cooperative Energy Management.

Reinforcement Learning Techniques in Optimizing Energy Systems

Approximating Energy Market Clearing and Bidding With Model-Based Reinforcement Learning

Reinforcement Learning Based Power Grid Day-Ahead Planning and AI-Assisted Control

Reinforcement Learning for Joint Design and Control of Battery-PV Systems

Deep Reinforcement Learning–based Approach for Optimizing Energy Conversion in Integrated Electrical and Heating System with Renewable Energy

Real-Time Integrated Learning and Decision-Making for Asset Networks