Abstract:Reinforcement learning (RL) is an artificial intelligence algorithm that can learn adaptive optimal control law online. In view of the fact that the previous control approaches were usually overly dependent on the model parameters of system, and most existing RL methods are based on state feedback, their application in actual industrial production is limited. Additionally, developing accurate process system models and ensuring the closed-loop system’s control performance is more challenging, as modern businesses place a premium on product quality and economic efficiency. As a result, this work introduces a novel data-driven two-dimensional (2D) off-policy Q -learning method based on output feedback is used to achieve optimal tracking control for batch process. First, the error between the actual output and the given set-point is extended to the system to ensure the good tracking performance. Second, by analyzing the relationship between the value function and the Q -function obtained from the 2D system’s performance index, the 2D Bellman equation is obtained in terms of output feedback that is independent of the model parameters. The optimal control problem can be effectively solved by the proposed method in this paper when the policy iteration is executed using only the measurement data of system along the batch and time directions. Following that, the proposed approach’s unbiasedness and convergence are strictly confirmed. Finally, the simulation results for the injection molding process demonstrate that the proposed method is capable of determining the optimal control law as the number of batches is growing increasingly.

Data-Driven H∞ Optimal Output Feedback Control for Linear Discrete-Time Systems Based on Off-Policy Q-Learning

An efficient data-based off-policy Q-learning algorithm for optimal output feedback control of linear systems

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

The Adaptive Optimal Output Feedback Tracking Control of Unknown Discrete-Time Linear Systems Using a Multistep Q-Learning Approach

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Output Feedback H∞ Control for Discrete Time Singularly Perturbed Systems with Markov Lossy Network: the Round-Robin-like Protocol Case

Output Feedback Adaptive Robust Learning Control of a Class of Nonlinear Systems with Periodic Disturbances

Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems Using Online Approximators

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS with Unidentified Exosystem Dynamics.

Optimal H∞ output feedback control for a class of nonlinear systems

Optimal dynamic output feedback control of unknown linear continuous-time systems by adaptive dynamic programming

H∞$$ {h}_{\infty } $$ Optimal Output Tracking Control for Markov Jump Systems: A Reinforcement Learning‐based Approach

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Output-feedback Q-learning for discrete-time linear H-infinity tracking control: A Stackelberg game approach

Output‐feedback Q‐learning for discrete‐time linear <i>H</i><sup>∞</sup> tracking control: A Stackelberg game approach

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Novel two-dimensional off-policy Q -learning method for output feedback optimal tracking control of batch process with unknown dynamics

Optimal Learning Output Tracking Control: A Model-Free Policy Optimization Method With Convergence Analysis

An Information-state based Approach to the Optimal Output Feedback Control of Nonlinear Systems

Data‐driven disturbance compensation control for discrete‐time systems based on reinforcement learning

Quadratic Tracking Control of Linear Stochastic Systems with Unknown Dynamics Using Average Off-Policy Q-Learning Method