Abstract:Reinforcement learning (RL) is an artificial intelligence algorithm that can learn adaptive optimal control law online. In view of the fact that the previous control approaches were usually overly dependent on the model parameters of system, and most existing RL methods are based on state feedback, their application in actual industrial production is limited. Additionally, developing accurate process system models and ensuring the closed-loop system’s control performance is more challenging, as modern businesses place a premium on product quality and economic efficiency. As a result, this work introduces a novel data-driven two-dimensional (2D) off-policy Q -learning method based on output feedback is used to achieve optimal tracking control for batch process. First, the error between the actual output and the given set-point is extended to the system to ensure the good tracking performance. Second, by analyzing the relationship between the value function and the Q -function obtained from the 2D system’s performance index, the 2D Bellman equation is obtained in terms of output feedback that is independent of the model parameters. The optimal control problem can be effectively solved by the proposed method in this paper when the policy iteration is executed using only the measurement data of system along the batch and time directions. Following that, the proposed approach’s unbiasedness and convergence are strictly confirmed. Finally, the simulation results for the injection molding process demonstrate that the proposed method is capable of determining the optimal control law as the number of batches is growing increasingly.

Optimal Tracking Control of Nonlinear Batch Processes with Unknown Dynamics Using Two-Dimensional Off-Policy Interleaved Q-learning Algorithm

Novel data-driven two-dimensional Q-learning for optimal tracking control of batch process with unknown dynamics

Optimal tracking control of batch processes with time-invariant state delay: Adaptive Q-learning with two-dimensional state and control policy

Novel two-dimensional off-policy Q -learning method for output feedback optimal tracking control of batch process with unknown dynamics

Off-policy two-dimensional reinforcement learning for optimal tracking control of batch processes with network-induced dropout and disturbances

Nonlinear Neuro-Optimal Tracking Control via Stable Iterative Q-Learning Algorithm

Improved Model-Free H∞ Control for Batch Processes Via Off-Policy 2D Game Q-learning

Iterative Q-learning-based Nonlinear Optimal Tracking Control

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS with Unidentified Exosystem Dynamics.

An ADDHP-based Q-learning Algorithm for Optimal Tracking Control of Linear Discrete-Time Systems with Unknown Dynamics

Data-driven Optimal Tracking Control for a Class of Affine Non-Linear Continuous-Time Systems with Completely Unknown Dynamics

Probabilistic Tracking Control for Non-Gaussian Stochastic Process Using Novel Iterative Learning Algorithms

Quadratic Tracking Control of Linear Stochastic Systems with Unknown Dynamics Using Average Off-Policy Q-Learning Method

Model-Free Optimal Tracking Design With Evolving Control Strategies via Q-Learning

Data-driven Optimal Tracking Control of Discrete-Time Multi-Agent Systems with Two-Stage Policy Iteration Algorithm.

Linear Quadratic Tracking Control of Unknown Systems: A Two-Phase Reinforcement Learning Method.

Two-dimensional model-free Q-learning-based output feedback fault-tolerant control for batch processes

Robust Optimal Tracking Control for Multiplayer Systems by Off‐policy Q‐learning Approach

Data-driven Approximate Optimal Tracking Control Schemes for Unknown Non-Affine Non-Linear Multi-Player Systems Via Adaptive Dynamic Programming

Optimal Tracking Control for Non-Zero-sum Games of Linear Discrete-Time Systems Via Off-Policy Reinforcement Learning

NN Adaptive Optimal Tracking Control for a Class of Uncertain Nonstrict Feedback Nonlinear Systems