Abstract:As an efficient way to integrate multiple distributed energy resources (DERs) and the user side, a microgrid is mainly faced with the problems of small-scale volatility, uncertainty, intermittency and demand-side uncertainty of DERs. The traditional microgrid has a single form and cannot meet the flexible energy dispatch between the complex demand side and the microgrid. In response to this problem, the overall environment of wind power, thermostatically controlled loads (TCLs), energy storage systems (ESSs), price-responsive loads and the main grid is proposed. Secondly, the centralized control of the microgrid operation is convenient for the control of the reactive power and voltage of the distributed power supply and the adjustment of the grid frequency. However, there is a problem in that the flexible loads aggregate and generate peaks during the electricity price valley. The existing research takes into account the power constraints of the microgrid and fails to ensure a sufficient supply of electric energy for a single flexible load. This paper considers the response priority of each unit component of TCLs and ESSs on the basis of the overall environment operation of the microgrid so as to ensure the power supply of the flexible load of the microgrid and save the power input cost to the greatest extent. Finally, the simulation optimization of the environment can be expressed as a Markov decision process (MDP) process. It combines two stages of offline and online operations in the training process. The addition of multiple threads with the lack of historical data learning leads to low learning efficiency. The asynchronous advantage actor-critic (Memory A3C, M-A3C) with the experience replay pool memory library is added to solve the data correlation and nonstatic distribution problems during training. The multithreaded working feature of M-A3C can efficiently learn the resource priority allocation on the demand side of the microgrid and improve the flexible scheduling of the demand side of the microgrid, which greatly reduces the input cost. Comparison of the researched cost optimization results with the results obtained with the proximal policy optimization (PPO) algorithm reveals that the proposed algorithm has better performance in terms of convergence and optimization economics.

Data driven frequency control of isolated microgrids based on priority experience replay soft deep reinforcement learning algorithm

Deep reinforcement learning for adaptive frequency control of island microgrid considering control performance and economy

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

Generating adversarial deep reinforcement learning -based frequency control of Island City microgrid considering generalization of scenarios

Research on priority scheduling strategy for smoothing power fluctuations of microgrid tie‐lines based on PER‐DDPG algorithm

Quantum-inspired deep reinforcement learning for adaptive frequency control of low carbon park island microgrid considering renewable energy sources

A Safe Policy Learning-Based Method for Decentralized and Economic Frequency Control in Isolated Networked-Microgrid Systems

A Data-Driven Energy Management Strategy Based on Deep Reinforcement Learning for Microgrid Systems

Deep Reinforcement Learning for Load Frequency Control in Isolated Microgrids: A Knowledge Aggregation Approach with Emphasis on Power Symmetry and Balance

Deep Meta-Reinforcement Learning Based Data-Driven Active Fault Tolerance Load Frequency Control for Islanded Microgrids Considering Internet of Things

Deep Reinforcement Learning Microgrid Optimization Strategy Considering Priority Flexible Demand Side

Distributed deep reinforcement learning for integrated generation‐control and power‐dispatch of interconnected power grid with various renewable units

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

Frequency regulation of multi-microgrid with shared energy storage based on deep reinforcement learning

Real-time Energy Management of a Low-Carbon Micro-Grid Based on Deep Reinforcement Learning

Data‐driven cooperative load frequency control method for microgrids using effective exploration‐distributed multi‐agent deep reinforcement learning

Prioritized experience replay based deep distributional reinforcement learning for battery operation in microgrids

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

Distributional Deep Reinforcement Learning-Based Emegercy Frequency Control

Coordinated Energy and Reserve Sharing of Isolated Microgrid Cluster using Deep Reinforcement Learning

A frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning