Abstract:Reinforcement learning (RL) is an artificial intelligence algorithm that can learn adaptive optimal control law online. In view of the fact that the previous control approaches were usually overly dependent on the model parameters of system, and most existing RL methods are based on state feedback, their application in actual industrial production is limited. Additionally, developing accurate process system models and ensuring the closed-loop system’s control performance is more challenging, as modern businesses place a premium on product quality and economic efficiency. As a result, this work introduces a novel data-driven two-dimensional (2D) off-policy Q -learning method based on output feedback is used to achieve optimal tracking control for batch process. First, the error between the actual output and the given set-point is extended to the system to ensure the good tracking performance. Second, by analyzing the relationship between the value function and the Q -function obtained from the 2D system’s performance index, the 2D Bellman equation is obtained in terms of output feedback that is independent of the model parameters. The optimal control problem can be effectively solved by the proposed method in this paper when the policy iteration is executed using only the measurement data of system along the batch and time directions. Following that, the proposed approach’s unbiasedness and convergence are strictly confirmed. Finally, the simulation results for the injection molding process demonstrate that the proposed method is capable of determining the optimal control law as the number of batches is growing increasingly.

Off-policy Q-learning-based Output Feedback Fault-tolerant Tracking Control of Industrial Processes

Off-policy Reinforcement Learning-Based Novel Model-Free Minmax Fault-Tolerant Tracking Control for Industrial Processes

H∞output Feedback Fault-Tolerant Control of Industrial Processes Based on Zero-Sum Games and Off-Policy Q-learning

I&I Adaptive Dynamic Feedback Fault-Tolerant Tracking Control of a Class of Nonaffine Systems with Nonlinearly Parameterized Faults

Two-dimensional model-free Q-learning-based output feedback fault-tolerant control for batch processes

Reinforcement Learning-Based Optimal Fault-Tolerant Tracking Control of Industrial Processes

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Novel two-dimensional off-policy Q -learning method for output feedback optimal tracking control of batch process with unknown dynamics

Operational Optimal Tracking Control for Industrial Multirate Systems Subject to Unknown Disturbances

Design of State Space Linear Quadratic Tracking Control Using GA Optimization for Batch Processes with Partial Actuator Failure

New Design of State Space Linear Quadratic Fault-Tolerant Tracking Control for Batch Processes with Partial Actuator Failure

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

The Adaptive Optimal Output Feedback Tracking Control of Unknown Discrete-Time Linear Systems Using a Multistep Q-Learning Approach

Fault-tolerant Control for Time-Delay Systems Via Output Dynamical Feedback

Novel data-driven two-dimensional Q-learning for optimal tracking control of batch process with unknown dynamics

Two-Dimensional Reinforcement Learning Model-Free Fault-Tolerant Control for Batch Processes Against Multi- Faults

Data-Driven Optimal Fault-Tolerant Control for Unknown Linear Systems

New Minmax Linear Quadratic Fault-Tolerant Tracking Control for Batch Processes

Event-Triggered Optimal Tracking Control for Strict-Feedback Nonlinear Systems With Non-Affine Nonlinear Faults

An efficient data-based off-policy Q-learning algorithm for optimal output feedback control of linear systems

Improved Infinite Horizon LQ Tracking Control for Injection Molding Process Against Partial Actuator Failures