Abstract:The electric power grid is changing from a traditional power system to a modern, smart, and integrated power system. Microgrids (MGs) play a vital role in combining distributed renewable energy resources (RESs) with traditional electric power systems. Intermittency, randomness, and volatility constitute the disadvantages of distributed RESs. MGs with high penetrations of renewable energy and random load demand cannot ignore these uncertainties, making it difficult to operate them effectively and economically. To realize the optimal scheduling of MGs, a real-time economic energy management strategy based on deep reinforcement learning (DRL) is proposed in this paper. Different from traditional model-based approaches, this strategy is learning based, and it has no requirements for an explicit model of uncertainty. Taking into account the uncertainties in RESs, load demand, and electricity prices, we formulate a Markov decision process for the real-time economic energy management problem of MGs. The objective is to minimize the daily operating cost of the system by scheduling controllable distributed generators and energy storage systems. In this paper, a deep deterministic policy gradient (DDPG) is introduced as a method for resolving the Markov decision process. The DDPG is a novel policy-based DRL approach with continuous state and action spaces. The DDPG is trained to learn the characteristics of uncertainties of the load, RES output, and electricity price using historical data from real power systems. The effectiveness of the proposed approach is validated through the designed simulation experiments. In the second experiment of our designed simulation, the proposed DRL method is compared to DQN, SAC, PPO, and MPC methods, and it is able to reduce the operating costs by 29.59%, 17.39%, 6.36%, and 9.55% on the June test set and 30.96%, 18.34%, 5.73%, and 10.16% on the November test set, respectively. The numerical results validate the practical value of the proposed DRL algorithm in addressing economic operation issues in MGs, as it demonstrates the algorithm's ability to effectively leverage the energy storage system to reduce the operating costs across a range of scenarios.

Rolling horizon wind-thermal unit commitment optimization based on deep reinforcement learning

Deep Reinforcement Learning-Based Scheduling Optimization of a Wind-solar Coupled CHP Unit System Considering Source-load Uncertainty

Deep Reinforcement Learning for Stochastic Dynamic Microgrid Energy Management

A deep reinforcement learning method for managing wind farm uncertainties through energy storage system control and external reserve purchasing

Deep Learning-Based Rolling Horizon Unit Commitment under Hybrid Uncertainties

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Hierarchical deep reinforcement learning for self-adaptive economic dispatch

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

Joint Energy Dispatch and Unit Commitment in Microgrids Based on Deep Reinforcement Learning

Real-Time Scheduling Method Based on Deep Reinforcement Learning for a Hybrid Wind-Solar-Storage Energy Generation System

Quantum Reinforcement Learning-Based Two-Stage Unit Commitment Framework for Enhanced Power Systems Robustness

A Data-driven Method for Fast AC Optimal Power Flow Solutions via Deep Reinforcement Learning

Deep reinforcement learning for real-time economic energy management of microgrid system considering uncertainties

Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning

Power balance control of RES integrated power system by deep reinforcement learning with optimized utilization rate of renewable energy

Economical Operation Strategy of an Integrated Energy System with Wind Power and Power to Gas Technology – a DRL‐based Approach

Reinforcement Learning Based Power Grid Day-Ahead Planning and AI-Assisted Control

Distributionally Robust Proximal Policy Optimization for Unit Commitment with Wind Power Uncertainty

Jointly improving energy efficiency and smoothing power oscillations of integrated offshore wind and photovoltaic power: a deep reinforcement learning approach

A Safe DRL Method for Fast Solution of Real-Time Optimal Power Flow

Deep Reinforcement Learning–based Approach for Optimizing Energy Conversion in Integrated Electrical and Heating System with Renewable Energy