Two-Dimensional Reinforcement Learning Model-Free Fault-Tolerant Control for Batch Processes Against Multi- Faults

Limin Wang,Linzhu Jia,Tao Zou,Ridong Zhang,Furong Gao
DOI: https://doi.org/10.1016/j.compchemeng.2024.108883
2025-01-01
Abstract:Aiming at the characteristics of batch process changing along with time and batch directions, the existence of unmodeled dynamics, and the partial failure of actuators or/and sensors, we propose a novel 2D reinforcement learning (RL) fault tolerant control strategy without considering model parameters. Firstly, a two-Dimensional (2D) augmented state space model and 2D Q function-based fault tolerant control (FTC) framework is established. The 2D Bellman equation can be acquired by analyzing the relationship between the 2D value function and the 2D Q function. Based on the extended model and Q-learning concept of RL, a data-driven FTTC independent of model parameters is designed, and a 2D data-driven Q-learning algorithm is proposed. Finally, taking the pressure holding phase in the injection process as the experimental object, the control effect is compared with that of the traditional model-based FTC, and better tracking performance and unbiasedness to the probing noise can be proved.
What problem does this paper attempt to address?