Abstract:A reinforcement learning (RL) based fault-tolerant control strategy is developed in this paper for wind turbine torque & pitch control under actuator & sensor faults subject to unknown system models. An incremental model-based heuristic dynamic programming (IHDP) approach, along with a critic-actor structure, is designed to enable fault-tolerance capability and achieve optimal control. Particularly, an incremental model is embedded in the critic-actor structure to quickly learn the potential system changes, such as faults, in real-time. Different from the current IHDP methods that need the intensive evaluation of the state and input matrices, only the input matrix of the incremental model is dynamically evaluated and updated by an online recursive least square estimation procedure in our proposed method. Such a design significantly enhances the online model evaluation efficiency and control performance, especially under faulty conditions. In addition, a value function and a target critic network are incorporated into the main critic-actor structure to improve our method’s learning effectiveness. Case studies for wind turbines under various working conditions are conducted based on the fatigue, aerodynamics, structures, and turbulence (FAST) simulator to demonstrate the proposed method’s solid fault-tolerance capability and adaptability. Note to Practitioners—This work achieves high-performance wind turbine control under unknown actuator & sensor faults. Such a task is still an open problem due to the complexity of turbine dynamics and potential uncertainties in practical situations. A novel data-driven and model-free control strategy based on reinforcement learning is proposed to handle these issues. The designed method can quickly capture the potential changes in the system and adjust its control policy in real-time, rendering strong adaptability and fault-tolerant abilities. It provides data-driven innovations for complex operational tasks of wind turbines and demonstrates the feasibility of applying reinforcement learning to handle fault-tolerant control problems. The proposed method has a generic structure and has the potential to be implemented in other renewable energy systems.

Residual Integral Inverse Reinforcement Learning for Intelligent Self-Healing Control of Unknown Systems with Actuator Faults

I&I Adaptive Fault-Tolerant Control of a Class of Nonaffine Systems with Nonlinearly Parameterised Faults

I&I Adaptive Dynamic Feedback Fault-Tolerant Tracking Control of a Class of Nonaffine Systems with Nonlinearly Parameterized Faults

Convergence Analysis of an Incremental Approach to Online Inverse Reinforcement Learning

Data-Driven Optimal Fault-Tolerant Control for Unknown Linear Systems

Adaptive Learning Based Fault Tolerant Control for Uncertain Nonlinear Systems

Adaptive Optimal Control For A Class Of Uncertain Systems With Saturating Actuators And External Disturbance Using Integral Reinforcement Learning

Adaptive Fuzzy Output-Constrained Control of Uncertain MISO Nonlinear Systems with Actuator Faults

Robust optimal control of the multi‐input systems with unknown disturbance based on adaptive integral reinforcement learning Q‐function

Reinforcement Learning-Based Optimal Fault-Tolerant Tracking Control of Industrial Processes

Adaptive Fault-Tolerant Robust Control for A Linear System with Adaptive Fault Identification

Integrated Adaptive Fault-Tolerant H Infinity Output Feedback Control with Adaptive Fault Identification

Adaptive Fault-tolerant Neural Control for Large-scale Systems with Actuator Faults

Data-based neural controls for an unknown continuous-time multi-input system with integral reinforcement

Wind Turbine Fault-Tolerant Control via Incremental Model-Based Reinforcement Learning

Online inverse reinforcement learning with unknown disturbances

A novel fault reconstruction and estimation approach for a class of systems subject to actuator and sensor faults under relaxed assumptions

Event-Driven Guaranteed Cost Control Design for Nonlinear Systems With Actuator Faults via Reinforcement Learning Algorithm

Integrating reinforcement learning with deterministic learning for fault diagnosis of nonlinear systems

Adaptive fuzzy fault‐tolerant control for cooperative output regulation with unknown nonlinear disturbances and actuator faults

Fault Diagnosis and Robust Fault-Tolerant Control for Linear Over-Actuated Systems