Abstract:A reinforcement learning (RL) based fault-tolerant control strategy is developed in this paper for wind turbine torque & pitch control under actuator & sensor faults subject to unknown system models. An incremental model-based heuristic dynamic programming (IHDP) approach, along with a critic-actor structure, is designed to enable fault-tolerance capability and achieve optimal control. Particularly, an incremental model is embedded in the critic-actor structure to quickly learn the potential system changes, such as faults, in real-time. Different from the current IHDP methods that need the intensive evaluation of the state and input matrices, only the input matrix of the incremental model is dynamically evaluated and updated by an online recursive least square estimation procedure in our proposed method. Such a design significantly enhances the online model evaluation efficiency and control performance, especially under faulty conditions. In addition, a value function and a target critic network are incorporated into the main critic-actor structure to improve our method’s learning effectiveness. Case studies for wind turbines under various working conditions are conducted based on the fatigue, aerodynamics, structures, and turbulence (FAST) simulator to demonstrate the proposed method’s solid fault-tolerance capability and adaptability. Note to Practitioners—This work achieves high-performance wind turbine control under unknown actuator & sensor faults. Such a task is still an open problem due to the complexity of turbine dynamics and potential uncertainties in practical situations. A novel data-driven and model-free control strategy based on reinforcement learning is proposed to handle these issues. The designed method can quickly capture the potential changes in the system and adjust its control policy in real-time, rendering strong adaptability and fault-tolerant abilities. It provides data-driven innovations for complex operational tasks of wind turbines and demonstrates the feasibility of applying reinforcement learning to handle fault-tolerant control problems. The proposed method has a generic structure and has the potential to be implemented in other renewable energy systems.

Reinforcement Learning-Based Optimal Fault-Tolerant Tracking Control of Industrial Processes

Off-policy Reinforcement Learning-Based Novel Model-Free Minmax Fault-Tolerant Tracking Control for Industrial Processes

Accelerating Reinforcement Learning with Local Data Enhancement for Process Control

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Operational Optimal Tracking Control for Industrial Multirate Systems Subject to Unknown Disturbances

Off-policy Q-learning-based Output Feedback Fault-tolerant Tracking Control of Industrial Processes

Two-Dimensional Reinforcement Learning Model-Free Fault-Tolerant Control for Batch Processes Against Multi- Faults

Data-Driven Optimal Fault-Tolerant Control for Unknown Linear Systems

H∞output Feedback Fault-Tolerant Control of Industrial Processes Based on Zero-Sum Games and Off-Policy Q-learning

Reinforcement-Learning-Based Tracking Control of Waste Water Treatment Process under Realistic System Conditions and Control Performance Requirements

Wind Turbine Fault-Tolerant Control via Incremental Model-Based Reinforcement Learning

Robust Dynamic Output Feedback Predictive Fault-Tolerant Control for Discrete Uncertain Systems

Constrained model predictive fault-tolerant control for nonlinear batch processes with time delay by integrating a LRF method and a switching strategy

New Design of State Space Linear Quadratic Fault-Tolerant Tracking Control for Batch Processes with Partial Actuator Failure

Event-Driven Guaranteed Cost Control Design for Nonlinear Systems With Actuator Faults via Reinforcement Learning Algorithm

Event-Triggered Optimal Tracking Control for Strict-Feedback Nonlinear Systems With Non-Affine Nonlinear Faults

Safe Transfer-Reinforcement-Learning-Based Optimal Control of Nonlinear Systems

A Model-Free Resilient Tracking Control Scheme for Non-Affine Nonlinear Continuous Systems with Uncertainties

Machine learning model‐based optimal tracking control of nonlinear affine systems with safety constraints

Probability-Based Robust Stochastic Predictive Fault-Tolerant Control for Industrial Processes With Actuator Failures and Interval Time-Varying Delays