Abstract:The electric power grid is changing from a traditional power system to a modern, smart, and integrated power system. Microgrids (MGs) play a vital role in combining distributed renewable energy resources (RESs) with traditional electric power systems. Intermittency, randomness, and volatility constitute the disadvantages of distributed RESs. MGs with high penetrations of renewable energy and random load demand cannot ignore these uncertainties, making it difficult to operate them effectively and economically. To realize the optimal scheduling of MGs, a real-time economic energy management strategy based on deep reinforcement learning (DRL) is proposed in this paper. Different from traditional model-based approaches, this strategy is learning based, and it has no requirements for an explicit model of uncertainty. Taking into account the uncertainties in RESs, load demand, and electricity prices, we formulate a Markov decision process for the real-time economic energy management problem of MGs. The objective is to minimize the daily operating cost of the system by scheduling controllable distributed generators and energy storage systems. In this paper, a deep deterministic policy gradient (DDPG) is introduced as a method for resolving the Markov decision process. The DDPG is a novel policy-based DRL approach with continuous state and action spaces. The DDPG is trained to learn the characteristics of uncertainties of the load, RES output, and electricity price using historical data from real power systems. The effectiveness of the proposed approach is validated through the designed simulation experiments. In the second experiment of our designed simulation, the proposed DRL method is compared to DQN, SAC, PPO, and MPC methods, and it is able to reduce the operating costs by 29.59%, 17.39%, 6.36%, and 9.55% on the June test set and 30.96%, 18.34%, 5.73%, and 10.16% on the November test set, respectively. The numerical results validate the practical value of the proposed DRL algorithm in addressing economic operation issues in MGs, as it demonstrates the algorithm's ability to effectively leverage the energy storage system to reduce the operating costs across a range of scenarios.

Consensus Based Distributed Reinforcement Learning For Nonconvex Economic Power Dispatch In Microgrids

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

Multi-Agent Reinforcement Learning-based Distributed Economic Dispatch Considering Network attacks and Uncertain Costs

Distributed Virtual Incremental Cost Consensus Algorithm For Economic Dispatch In A Microgrid

A Distributed Consensus Based Algorithm for Economic Dispatch over Time‐varying Digraphs

Multi-Agent System Based Optimal Power Dispatch Algorithm for Microgrid

Distributed Economic Dispatch in Microgrids Based on Cooperative Reinforcement Learning

Microgrid Energy Management Using Improved Reinforcement Learning with Quadratic Programming

A Deep Reinforcement Learning Method for Economic Power Dispatch of Microgrid in OPAL-RT Environment

Multiagent-Based Reinforcement Learning for Optimal Reactive Power Dispatch.

Distributed Multiagent Reinforcement Learning With Action Networks for Dynamic Economic Dispatch

Distributed Online Dispatch for Microgrids Using Hierarchical Reinforcement Learning Embedded with Operation Knowledge

Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

Distributed Consensus-Based Economic Dispatch for Power System over Directed Network*

A Distributed Learning-Based Economic Dispatch Approach of Microgrids Considering Operational Mode Switch

Distributed Economic Dispatch for Energy Internet Based on Multi-Agent Consensus Control

A Fully Distributed ADP Algorithm for Real-time Economic Dispatch of Microgrid

Deep reinforcement learning for real-time economic energy management of microgrid system considering uncertainties

Dynamic Energy Dispatch Based on Deep Reinforcement Learning in IoT-Driven Smart Isolated Microgrids