Abstract:The construction of microgrid has fully promoted the large-scale access of distributed power and the rapid development of electric vehicles (EVs). And microgrid system faces operational issues such as strong random disturbances from distributed power sources and loads, as well as unexpected events during operation. These issues can lead to unstable microgrid frequency, excessive discharge of EVs, and increased control costs. To solve the issues, a frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning (EDRL) is proposed in this paper. First, consider the effect of the vehicle-to-grid (V2G) process on the minimum time required to fully charge an EV, and the impact of the output distribution of micro turbines (MTs) on the regulation cost, and based on the coupling relationship between generator terminal voltage regulation and system frequency control, a multimicrogrid comprehensive control model is constructed. Second, in order to deal with engineering tasks with deceptive rewards and sparse rewards such as integrated frequency control, evolutionary algorithms and deep reinforcement learning algorithms are combined to effectively assist the training process to jump out of local optimal solutions and approach optimal control strategies. Meanwhile, the algorithm is improved based on the novelty search and intelligent partition strategy, so that the strategy has better convergence characteristics and can reduce the cost of information transmission and computational complexity under the premise of ensuring the control effect. Furthermore, the state space, action space and reward function of the controller are defined. Finally, the simulation results show that the proposed controller has the ability of coordinated control, can effectively reduce the adjustment cost of MT unit and the unnecessary discharging of EVs while ensuring the frequency regulation requirements of each submicrogrid, which is far superior to PID control, fuzzy control, and traditional deep reinforcement learning control.

Cooperative grid frequency control under asymmetric V2G capacity via switched integral reinforcement learning

An Adaptive V2G Capacity-based Frequency Regulation Scheme with Integral Reinforcement Learning against DoS Attacks

Optimal vehicle-to-grid control for supplementary frequency regulation using deep reinforcement learning

Vehicle-to-Grid Control for Supplementary Frequency Regulation Considering Charging Demands

Coordination of PV Smart Inverters Using Deep Reinforcement Learning for Grid Voltage Regulation

A frequency cooperative control strategy for multimicrogrids with EVs based on improved evolutionary-deep reinforcement learning

Online EVs Vehicle-to-Grid Scheduling Coordinated with Multi-Energy Microgrids: A Deep Reinforcement Learning-Based Approach

An Inertia Adjustment Control Strategy of Grid-Forming Electric Vehicle for V2G Application

Decentralized Vehicle-to-Grid Control for Primary Frequency Regulation Considering Charging Demands

Coordinated Control for Large-Scale EV Charging Facilities and Energy Storage Devices Participating in Frequency Regulation

Transfer Deep Reinforcement Learning-based Large-scale V2G Continuous Charging Coordination with Renewable Energy Sources

EV Dispatch Control for Supplementary Frequency Regulation Considering the Expectation of EV Owners

Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks

Coordinated Vehicle-to-Grid Control to Participate in Secondary Frequency Regulation

Electric vehicle charging and discharging scheduling strategy based on dynamic electricity price

Distributed deep reinforcement learning for integrated generation‐control and power‐dispatch of interconnected power grid with various renewable units

Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning

A Distributed Dynamic Inertia-droop Control Strategy Based on Multi-Agent Deep Reinforcement Learning for Multiple Paralleled VSGs

Distributed Vehicle-to-Grid (V2G) Coordination: an Efficient and Cost-Effective Framework Based on Multilevel Game Theory

Deep-Reinforcement-Learning-Based Vehicle-to-Grid Operation Strategies for Managing Solar Power Generation Forecast Errors