Abstract:The electric power grid is changing from a traditional power system to a modern, smart, and integrated power system. Microgrids (MGs) play a vital role in combining distributed renewable energy resources (RESs) with traditional electric power systems. Intermittency, randomness, and volatility constitute the disadvantages of distributed RESs. MGs with high penetrations of renewable energy and random load demand cannot ignore these uncertainties, making it difficult to operate them effectively and economically. To realize the optimal scheduling of MGs, a real-time economic energy management strategy based on deep reinforcement learning (DRL) is proposed in this paper. Different from traditional model-based approaches, this strategy is learning based, and it has no requirements for an explicit model of uncertainty. Taking into account the uncertainties in RESs, load demand, and electricity prices, we formulate a Markov decision process for the real-time economic energy management problem of MGs. The objective is to minimize the daily operating cost of the system by scheduling controllable distributed generators and energy storage systems. In this paper, a deep deterministic policy gradient (DDPG) is introduced as a method for resolving the Markov decision process. The DDPG is a novel policy-based DRL approach with continuous state and action spaces. The DDPG is trained to learn the characteristics of uncertainties of the load, RES output, and electricity price using historical data from real power systems. The effectiveness of the proposed approach is validated through the designed simulation experiments. In the second experiment of our designed simulation, the proposed DRL method is compared to DQN, SAC, PPO, and MPC methods, and it is able to reduce the operating costs by 29.59%, 17.39%, 6.36%, and 9.55% on the June test set and 30.96%, 18.34%, 5.73%, and 10.16% on the November test set, respectively. The numerical results validate the practical value of the proposed DRL algorithm in addressing economic operation issues in MGs, as it demonstrates the algorithm's ability to effectively leverage the energy storage system to reduce the operating costs across a range of scenarios.

A multi-agent deep reinforcement learning approach for temporally coordinated demand response in microgrids

Integrated Demand Response in Multi-Energy Microgrids: A Deep Reinforcement Learning-Based Approach

Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework

Enhancing Grid-Interactive Buildings Demand Response: Sequential Update-Based Multi-Agent Deep Reinforcement Learning Approach

Price-Based Residential Demand Response Management in Smart Grids: A Reinforcement Learning-Based Approach

Optimal demand response based dynamic pricing strategy via Multi-Agent Federated Twin Delayed Deep Deterministic policy gradient algorithm

Multi-agent deep reinforcement learning-based optimal energy management for grid-connected multiple energy carrier microgrids

Micro-grid Oriented Active Demand Response Optimization Scheme

A Real-time Demand Response Strategy of Home Energy Management by Using Distributed Deep Reinforcement Learning.

Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

Deep reinforcement learning for real-time economic energy management of microgrid system considering uncertainties

Multi-agent DRL-based Data-Driven Approach for PEVs Charging/discharging Scheduling in Smart Grid.

Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management

Asynchronous Multi-Agent Reinforcement Learning-Based Framework for Bi-Level Noncooperative Game-Theoretic Demand Response

Reinforcement Learning Based Demand Response Approach Incorporating Thermostatically Controlled Loads and Battery Loads

Cooperative price-based demand response program for multiple aggregators based on multi-agent reinforcement learning and shapley-value

Intelligent demand response resource trading using deep reinforcement learning

Physical Informed-Inspired Deep Reinforcement Learning Based Bi-Level Programming for Microgrid Scheduling

Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning

Deep Reinforcement Learning Based Bi-layer Optimal Scheduling for Microgrids Considering Flexible Load Control