Abstract:Wind farm (WF) is gradually required to provide inertial support during frequency events nowadays. Existing WF inertial control methods have exhibited limitations in parameter estimation and optimal control parameter tuning under variable operating conditions. Moreover, traditional model-based methods suffer from computational efficiency problems due to their reliance on repeated iterations or derivative computations in optimization process. Thus, this paper proposes an inertial control method for large-scale WF based on hierarchical distributed hybrid deep-reinforcement learning (HDH-DRL). Firstly, the control objectives for the inertial control are defined, and are decomposed on the basis of wind turbines (WTs) division. Next, this paper presents a HDH-DRL based inertial control framework consisting of two levels. The upper-level control is achieved by the hybrid multi-agent DRL (HMA-DRL) algorithm with hybrid action exploration mechanism and multi-agent coordination. The consensus based lower-level control aims to achieve the consensus convergence of the control through the locally distributed interaction among WTs. Finally, the inertial control processes of the model are exhibited; the influences of DRL algorithms on the control are discussed; the computational speed and solution accuracy of the proposed method are compared with the model-based methods; the multi-scenario applicability and performance in local communication failures of the model are analyzed. The results demonstrate that the proposed method significantly improves the active power response and frequency stability, achieving rapid consensus convergence with a power deviation below 0.3%, surpassing traditional control methods by at least 0.2%; also, it demonstrates superior performance across various scenarios, including transient and steady-state conditions, with frequency enhancements up to 0.30% and exceptional stability under different wind conditions and communication disruptions.

Reinforcement Learning-Based Wind Farm Control: Toward Large Farm Applications via Automatic Grouping and Transfer Learning

Intelligent Wind Farm Control via Grouping-Based Reinforcement Learning

A grouping strategy for reinforcement learning-based collective yaw control of wind farms

Reinforcement Learning-Based Multiobjective Control of Grid-Connected Wind Farms

A Multiagent Reinforcement Learning Approach for Wind Farm Frequency Control

An inertial control method for large-scale wind farm based on hierarchical distributed hybrid deep-reinforcement learning

Cooperative Wind Farm Control With Deep Reinforcement Learning and Knowledge-Assisted Learning

Wind Farm Power Generation Control Via Double-Network-Based Deep Reinforcement Learning

Collective Large-scale Wind Farm Multivariate Power Output Control Based on Hierarchical Communication Multi-Agent Proximal Policy Optimization

Deep Reinforcement Learning Based Wind Farm Cluster Reactive Power Optimization Considering Network Switching

Reinforcement learning for wind-farm flow control: Current state and future actions

Intelligent Fault-Tolerant Active Power Control Using Reinforcement Learning for Offshore Wind Farms

Deep learning-aided model predictive control of wind farms for AGC considering the dynamic wake effect

Jointly improving energy efficiency and smoothing power oscillations of integrated offshore wind and photovoltaic power: a deep reinforcement learning approach

Data-Driven Wind Farm Volt/VAR Control Based on Deep Reinforcement Learning

Reinforcement learning-enhanced genetic algorithm for wind farm layout optimization

Rolling horizon wind-thermal unit commitment optimization based on deep reinforcement learning

Proof-of-concept of a reinforcement learning framework for wind farm energy capture maximization in time-varying wind

Model-Free Optimization Scheme for Efficiency Improvement of Wind Farm Using Decentralized Reinforcement Learning

A teaching-learning-based optimization algorithm with reinforcement learning to address wind farm layout optimization problem

Coordinated control of wind turbine and hybrid energy storage system based on multi-agent deep reinforcement learning for wind power smoothing