Abstract:The electric power grid is changing from a traditional power system to a modern, smart, and integrated power system. Microgrids (MGs) play a vital role in combining distributed renewable energy resources (RESs) with traditional electric power systems. Intermittency, randomness, and volatility constitute the disadvantages of distributed RESs. MGs with high penetrations of renewable energy and random load demand cannot ignore these uncertainties, making it difficult to operate them effectively and economically. To realize the optimal scheduling of MGs, a real-time economic energy management strategy based on deep reinforcement learning (DRL) is proposed in this paper. Different from traditional model-based approaches, this strategy is learning based, and it has no requirements for an explicit model of uncertainty. Taking into account the uncertainties in RESs, load demand, and electricity prices, we formulate a Markov decision process for the real-time economic energy management problem of MGs. The objective is to minimize the daily operating cost of the system by scheduling controllable distributed generators and energy storage systems. In this paper, a deep deterministic policy gradient (DDPG) is introduced as a method for resolving the Markov decision process. The DDPG is a novel policy-based DRL approach with continuous state and action spaces. The DDPG is trained to learn the characteristics of uncertainties of the load, RES output, and electricity price using historical data from real power systems. The effectiveness of the proposed approach is validated through the designed simulation experiments. In the second experiment of our designed simulation, the proposed DRL method is compared to DQN, SAC, PPO, and MPC methods, and it is able to reduce the operating costs by 29.59%, 17.39%, 6.36%, and 9.55% on the June test set and 30.96%, 18.34%, 5.73%, and 10.16% on the November test set, respectively. The numerical results validate the practical value of the proposed DRL algorithm in addressing economic operation issues in MGs, as it demonstrates the algorithm's ability to effectively leverage the energy storage system to reduce the operating costs across a range of scenarios.

Application-oriented assessment of grid-connected PV-battery system with deep reinforcement learning in buildings considering electricity price dynamics

Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control

Operational Optimization for Off-Grid Renewable Building Energy System Using Deep Reinforcement Learning

Operational Optimization for the Grid-Connected Residential Photovoltaic-Battery System Using Model-Based Reinforcement Learning

Comparative Analysis of Reinforcement Learning Approaches for Multi-Objective Optimization in Residential Hybrid Energy Systems

Learning a local trading strategy: deep reinforcement learning for grid-scale renewable energy integration

Digital Twin-Based Economic Assessment of Solar Energy in Smart Microgrids Using Reinforcement Learning Technique

Price-Based Residential Demand Response Management in Smart Grids: A Reinforcement Learning-Based Approach

Deep reinforcement learning for real-time economic energy management of microgrid system considering uncertainties

Physical-model-free intelligent energy management for a grid-connected hybrid wind-microturbine-PV-EV energy system via deep reinforcement learning approach

Electric Vehicle Charging/Discharging Control Method Based on Deep Reinforcement Learning for Photovoltaic Energy Storage Stations

Dynamic Demand-Aware Power Grid Intelligent Pricing Algorithm Based on Deep Reinforcement Learning

Online Scheduling of PV and Energy Storage System Based on Deep Reinforcement Learning

Optimizing a domestic battery and solar photovoltaic system with deep reinforcement learning

Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices

Coordination of PV Smart Inverters Using Deep Reinforcement Learning for Grid Voltage Regulation

Deep Reinforcement Learning for Real-Time Energy Management in Smart Home

Deep reinforcement learning-based scheduling for integrated energy system utilizing retired electric vehicle battery energy storage

Operation of Distributed Battery Considering Demand Response Using Deep Reinforcement Learning in Grid Edge Control

Deep Reinforcement Learning for Charging Electric Vehicles with Uncertain Photovoltaic Power

Reinforcement Learning Based Demand Response Approach Incorporating Thermostatically Controlled Loads and Battery Loads