Abstract:The construction of microgrid has fully promoted the large-scale access of distributed power and the rapid development of electric vehicles (EVs). And microgrid system faces operational issues such as strong random disturbances from distributed power sources and loads, as well as unexpected events during operation. These issues can lead to unstable microgrid frequency, excessive discharge of EVs, and increased control costs. To solve the issues, a frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning (EDRL) is proposed in this paper. First, consider the effect of the vehicle-to-grid (V2G) process on the minimum time required to fully charge an EV, and the impact of the output distribution of micro turbines (MTs) on the regulation cost, and based on the coupling relationship between generator terminal voltage regulation and system frequency control, a multimicrogrid comprehensive control model is constructed. Second, in order to deal with engineering tasks with deceptive rewards and sparse rewards such as integrated frequency control, evolutionary algorithms and deep reinforcement learning algorithms are combined to effectively assist the training process to jump out of local optimal solutions and approach optimal control strategies. Meanwhile, the algorithm is improved based on the novelty search and intelligent partition strategy, so that the strategy has better convergence characteristics and can reduce the cost of information transmission and computational complexity under the premise of ensuring the control effect. Furthermore, the state space, action space and reward function of the controller are defined. Finally, the simulation results show that the proposed controller has the ability of coordinated control, can effectively reduce the adjustment cost of MT unit and the unnecessary discharging of EVs while ensuring the frequency regulation requirements of each submicrogrid, which is far superior to PID control, fuzzy control, and traditional deep reinforcement learning control.

Data‐driven cooperative load frequency control method for microgrids using effective exploration‐distributed multi‐agent deep reinforcement learning

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

A Multi-Agent Deep Reinforcement Learning-Based “octopus” Cooperative Load Frequency Control for an Interconnected Grid with Various Renewable Units

Deep reinforcement learning for adaptive frequency control of island microgrid considering control performance and economy

Coordinated load frequency control of multi-area integrated energy system using multi-agent deep reinforcement learning

Generating adversarial deep reinforcement learning -based frequency control of Island City microgrid considering generalization of scenarios

Deep Reinforcement Learning for Load Frequency Control in Isolated Microgrids: A Knowledge Aggregation Approach with Emphasis on Power Symmetry and Balance

Deep Meta-Reinforcement Learning Based Data-Driven Active Fault Tolerance Load Frequency Control for Islanded Microgrids Considering Internet of Things

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Data-Driven Load Frequency Control Based on Multi-Agent Reinforcement Learning With Attention Mechanism

Multiagent deep meta reinforcement learning for sea computing-based energy management of interconnected grids considering renewable energy sources in sustainable cities

Grid-area coordinated load frequency control strategy using large-scale multi-agent deep reinforcement learning

Demand-Side Frequency Control in Power Network Based on Multi-Agent Deep Reinforcement Learning

Distributed quantum multiagent deep meta reinforcement learning for area autonomy energy management of a multiarea microgrid

Evolutionary Multi Agent Deep Meta Reinforcement Learning Method for Swarm Intelligence Energy Management of Isolated Multi Area Microgrid with Internet of Things

A frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning

Model-Free Load Frequency Control Based on Multi-Agent Deep Reinforcement Learning

Distributed deep reinforcement learning for integrated generation‐control and power‐dispatch of interconnected power grid with various renewable units

Frequency Control for Islanded AC Microgrid Based on Deep Reinforcement Learning

Quantum-inspired deep reinforcement learning for adaptive frequency control of low carbon park island microgrid considering renewable energy sources

Distributed Economic Dispatch in Microgrids Based on Cooperative Reinforcement Learning