Abstract:Wind farm (WF) is gradually required to provide inertial support during frequency events nowadays. Existing WF inertial control methods have exhibited limitations in parameter estimation and optimal control parameter tuning under variable operating conditions. Moreover, traditional model-based methods suffer from computational efficiency problems due to their reliance on repeated iterations or derivative computations in optimization process. Thus, this paper proposes an inertial control method for large-scale WF based on hierarchical distributed hybrid deep-reinforcement learning (HDH-DRL). Firstly, the control objectives for the inertial control are defined, and are decomposed on the basis of wind turbines (WTs) division. Next, this paper presents a HDH-DRL based inertial control framework consisting of two levels. The upper-level control is achieved by the hybrid multi-agent DRL (HMA-DRL) algorithm with hybrid action exploration mechanism and multi-agent coordination. The consensus based lower-level control aims to achieve the consensus convergence of the control through the locally distributed interaction among WTs. Finally, the inertial control processes of the model are exhibited; the influences of DRL algorithms on the control are discussed; the computational speed and solution accuracy of the proposed method are compared with the model-based methods; the multi-scenario applicability and performance in local communication failures of the model are analyzed. The results demonstrate that the proposed method significantly improves the active power response and frequency stability, achieving rapid consensus convergence with a power deviation below 0.3%, surpassing traditional control methods by at least 0.2%; also, it demonstrates superior performance across various scenarios, including transient and steady-state conditions, with frequency enhancements up to 0.30% and exceptional stability under different wind conditions and communication disruptions.

Control of a Wave Energy Converter Using Model-Free Deep Reinforcement Learning

Design and Tank Testing of Reinforcement Learning Control for Wave Energy Converters

Environmental-Sensing and adaptive optimization of wave energy converter based on deep reinforcement learning and computational fluid dynamics

Optimal Control for AWS-Based Wave Energy Conversion System

An Active-Controlled Heaving Plate Breakwater Trained by an Intelligent Framework Based on Deep Reinforcement Learning

Experimental Implementation of Reinforcement Learning Applied to Maximise Energy from a Wave Energy Converter

Sail-rudder Collaborative Control of Unmanned Sailboat Based on Reinforcement Learning

A Novel Model Predictive Control Formulation for Wave Energy Converters Based on the Reactive Rollout Method

Model-Based Reinforcement Learning for Control of Strongly-Disturbed Unsteady Aerodynamic Flows

Application of Deep Reinforcement Learning Control of an Inverted Hydraulic Pendulum

Skip Training for Multi-Agent Reinforcement Learning Controller for Industrial Wave Energy Converters

Jointly improving energy efficiency and smoothing power oscillations of integrated offshore wind and photovoltaic power: a deep reinforcement learning approach

Model predictive control of wave energy converters

Real-Time Model-Free Deep Reinforcement Learning for Force Control of a Series Elastic Actuator

Deep reinforcement learning with reward shaping for tracking control and vibration suppression of flexible link manipulator

Data-Driven Performance-Prescribed Reinforcement Learning Control of an Unmanned Surface Vehicle

Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Deep Reinforcement Learning Based Controller for Active Heave Compensation

Adaptive optimization of wave energy conversion in oscillatory wave surge converters via SPH simulation and deep reinforcement learning

An inertial control method for large-scale wind farm based on hierarchical distributed hybrid deep-reinforcement learning