Abstract:Wind farm (WF) is gradually required to provide inertial support during frequency events nowadays. Existing WF inertial control methods have exhibited limitations in parameter estimation and optimal control parameter tuning under variable operating conditions. Moreover, traditional model-based methods suffer from computational efficiency problems due to their reliance on repeated iterations or derivative computations in optimization process. Thus, this paper proposes an inertial control method for large-scale WF based on hierarchical distributed hybrid deep-reinforcement learning (HDH-DRL). Firstly, the control objectives for the inertial control are defined, and are decomposed on the basis of wind turbines (WTs) division. Next, this paper presents a HDH-DRL based inertial control framework consisting of two levels. The upper-level control is achieved by the hybrid multi-agent DRL (HMA-DRL) algorithm with hybrid action exploration mechanism and multi-agent coordination. The consensus based lower-level control aims to achieve the consensus convergence of the control through the locally distributed interaction among WTs. Finally, the inertial control processes of the model are exhibited; the influences of DRL algorithms on the control are discussed; the computational speed and solution accuracy of the proposed method are compared with the model-based methods; the multi-scenario applicability and performance in local communication failures of the model are analyzed. The results demonstrate that the proposed method significantly improves the active power response and frequency stability, achieving rapid consensus convergence with a power deviation below 0.3%, surpassing traditional control methods by at least 0.2%; also, it demonstrates superior performance across various scenarios, including transient and steady-state conditions, with frequency enhancements up to 0.30% and exceptional stability under different wind conditions and communication disruptions.

A Multiagent Reinforcement Learning Approach for Wind Farm Frequency Control

Reinforcement Learning-Based Inertia and Droop Control for Wind Farm Frequency Regulation

Reinforcement Learning-Based Multiobjective Control of Grid-Connected Wind Farms

Reinforcement Learning-Based Wind Farm Control: Toward Large Farm Applications via Automatic Grouping and Transfer Learning

Coordinated control of wind turbine and hybrid energy storage system based on multi-agent deep reinforcement learning for wind power smoothing

Demand-Side Frequency Control in Power Network Based on Multi-Agent Deep Reinforcement Learning

Cooperative Wind Farm Control With Deep Reinforcement Learning and Knowledge-Assisted Learning

Multiagent deep meta reinforcement learning for sea computing-based energy management of interconnected grids considering renewable energy sources in sustainable cities

Data-Driven Load Frequency Control Based on Multi-Agent Reinforcement Learning With Attention Mechanism

Fast Frequency Response Using Reinforcement Learning-Controlled Wind Turbines.

Intelligent Fault-Tolerant Active Power Control Using Reinforcement Learning for Offshore Wind Farms

Reactive-Voltage Coordinated Control of Offshore Wind Farm Based on Deep Reinforcement Learning

A grouping strategy for reinforcement learning-based collective yaw control of wind farms

A Multi-Agent Deep Reinforcement Learning-Based “octopus” Cooperative Load Frequency Control for an Interconnected Grid with Various Renewable Units

Collective Large-scale Wind Farm Multivariate Power Output Control Based on Hierarchical Communication Multi-Agent Proximal Policy Optimization

Model-Free Load Frequency Control Based on Multi-Agent Deep Reinforcement Learning

An inertial control method for large-scale wind farm based on hierarchical distributed hybrid deep-reinforcement learning

Intelligent Wind Farm Control via Grouping-Based Reinforcement Learning

Distributed Adaptive Leaderless Consensus Control of Wind Farm for Fast Frequency Support

A Frequency Control Strategy Considering Large Scale Wind Power Cluster Integration Based on Distributed Model Predictive Control

Data-Driven Model Predictive Control Method for Wind Farms to Provide Frequency Support