Abstract:The electric power grid is changing from a traditional power system to a modern, smart, and integrated power system. Microgrids (MGs) play a vital role in combining distributed renewable energy resources (RESs) with traditional electric power systems. Intermittency, randomness, and volatility constitute the disadvantages of distributed RESs. MGs with high penetrations of renewable energy and random load demand cannot ignore these uncertainties, making it difficult to operate them effectively and economically. To realize the optimal scheduling of MGs, a real-time economic energy management strategy based on deep reinforcement learning (DRL) is proposed in this paper. Different from traditional model-based approaches, this strategy is learning based, and it has no requirements for an explicit model of uncertainty. Taking into account the uncertainties in RESs, load demand, and electricity prices, we formulate a Markov decision process for the real-time economic energy management problem of MGs. The objective is to minimize the daily operating cost of the system by scheduling controllable distributed generators and energy storage systems. In this paper, a deep deterministic policy gradient (DDPG) is introduced as a method for resolving the Markov decision process. The DDPG is a novel policy-based DRL approach with continuous state and action spaces. The DDPG is trained to learn the characteristics of uncertainties of the load, RES output, and electricity price using historical data from real power systems. The effectiveness of the proposed approach is validated through the designed simulation experiments. In the second experiment of our designed simulation, the proposed DRL method is compared to DQN, SAC, PPO, and MPC methods, and it is able to reduce the operating costs by 29.59%, 17.39%, 6.36%, and 9.55% on the June test set and 30.96%, 18.34%, 5.73%, and 10.16% on the November test set, respectively. The numerical results validate the practical value of the proposed DRL algorithm in addressing economic operation issues in MGs, as it demonstrates the algorithm's ability to effectively leverage the energy storage system to reduce the operating costs across a range of scenarios.

A hybrid deep learning-based online energy management scheme for industrial microgrid

Multi-energy Microgrid Robust Energy Management with a Novel Decision-Making Strategy

A hierarchical energy management strategy for grid-connected microgrid

Deep reinforcement learning for real-time economic energy management of microgrid system considering uncertainties

Renewable-based microgrids' energy management using smart deep learning techniques: Realistic digital twin case

Data-driven optimal energy management for a wind-solar-diesel-battery-reverse osmosis hybrid energy system using a deep reinforcement learning approach

Physical-model-free intelligent energy management for a grid-connected hybrid wind-microturbine-PV-EV energy system via deep reinforcement learning approach

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

A Microgrid Energy Management System and Risk Management Under an Electricity Market Environment

An Optimal Energy Storage Control Strategy for Grid-connected Microgrids

A Hybrid Approach for Home Energy Management with Imitation Learning and Online Optimization

Microgrid Energy Management System With Embedded Deep Learning Forecaster and Combined Optimizer

Online Distributed Neurodynamic Optimization for Energy Management of Renewable Energy Grids

Online Optimal Power Scheduling of a Microgrid via Imitation Learning

An optimal solutions-guided deep reinforcement learning approach for online energy storage control

Distributed Real-Time Energy Management in Data Center Microgrids

Double Deep Q-learning Based Real-Time Optimization Strategy for Microgrids

A novel rolling optimization strategy considering grid-connected power fluctuations smoothing for renewable energy microgrids

Online energy generation scheduling for microgrids with intermittent energy sources and co-generation

Towards Optimal Energy Management of Microgrids with a Realistic Model

Data-Driven Risk-Adjusted Robust Energy Management for Microgrids Integrating Demand Response Aggregator and Renewable Energies