Abstract:The construction of microgrid has fully promoted the large-scale access of distributed power and the rapid development of electric vehicles (EVs). And microgrid system faces operational issues such as strong random disturbances from distributed power sources and loads, as well as unexpected events during operation. These issues can lead to unstable microgrid frequency, excessive discharge of EVs, and increased control costs. To solve the issues, a frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning (EDRL) is proposed in this paper. First, consider the effect of the vehicle-to-grid (V2G) process on the minimum time required to fully charge an EV, and the impact of the output distribution of micro turbines (MTs) on the regulation cost, and based on the coupling relationship between generator terminal voltage regulation and system frequency control, a multimicrogrid comprehensive control model is constructed. Second, in order to deal with engineering tasks with deceptive rewards and sparse rewards such as integrated frequency control, evolutionary algorithms and deep reinforcement learning algorithms are combined to effectively assist the training process to jump out of local optimal solutions and approach optimal control strategies. Meanwhile, the algorithm is improved based on the novelty search and intelligent partition strategy, so that the strategy has better convergence characteristics and can reduce the cost of information transmission and computational complexity under the premise of ensuring the control effect. Furthermore, the state space, action space and reward function of the controller are defined. Finally, the simulation results show that the proposed controller has the ability of coordinated control, can effectively reduce the adjustment cost of MT unit and the unnecessary discharging of EVs while ensuring the frequency regulation requirements of each submicrogrid, which is far superior to PID control, fuzzy control, and traditional deep reinforcement learning control.

Frequency Control for Islanded AC Microgrid Based on Deep Reinforcement Learning

Deep reinforcement learning for adaptive frequency control of island microgrid considering control performance and economy

AdapSafe: Adaptive and Safe-Certified Deep Reinforcement Learning-Based Frequency Control for Carbon-Neutral Power Systems.

Generating adversarial deep reinforcement learning -based frequency control of Island City microgrid considering generalization of scenarios

Highly transferable adversarial attack against deep‐reinforcement‐learning‐based frequency control

Frequency regulation of multi-microgrid with shared energy storage based on deep reinforcement learning

Quantum-inspired deep reinforcement learning for adaptive frequency control of low carbon park island microgrid considering renewable energy sources

Data driven frequency control of isolated microgrids based on priority experience replay soft deep reinforcement learning algorithm

Deep Meta-Reinforcement Learning Based Data-Driven Active Fault Tolerance Load Frequency Control for Islanded Microgrids Considering Internet of Things

Distributional Deep Reinforcement Learning-Based Emegercy Frequency Control

A Safe Policy Learning-Based Method for Decentralized and Economic Frequency Control in Isolated Networked-Microgrid Systems

A frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning

Deep Reinforcement Learning for Load Frequency Control in Isolated Microgrids: A Knowledge Aggregation Approach with Emphasis on Power Symmetry and Balance

Data‐driven cooperative load frequency control method for microgrids using effective exploration‐distributed multi‐agent deep reinforcement learning

A Multiagent Reinforcement Learning Approach for Wind Farm Frequency Control

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

Automatic generation control of is-landed micro-grid using integral reinforcement learning-based adaptive optimal control strategy

Optimizing the Post-Disaster Control of Islanded Microgrid: A Multi-Agent Deep Reinforcement Learning Approach.

Deep Deterministic Policy Gradient Reinforcement Learning Based Adaptive PID Load Frequency Control of an AC Micro-Grid

Distributed deep reinforcement learning for integrated generation‐control and power‐dispatch of interconnected power grid with various renewable units

Real-time Energy Management of a Low-Carbon Micro-Grid Based on Deep Reinforcement Learning