Abstract:Distributed parameter systems (DPSs) frequently appear in industrial manufacturing processes, with complex characteristics such as time–space coupling, nonlinearity, infinite dimension, uncertainty and so on, which is full of challenges to the modeling of the system. At present, most DPS modeling methods are offline. When the internal parameters or external environment of DPS change, the offline model is incapable of accurately representing the dynamic attributes of the real system. Establishing an online model for DPS that accurately reflects the real-time dynamics of the system is very important. In this paper, the idea of reinforcement learning is creatively integrated into the three-dimensional (3D) fuzzy model and a reinforcement learning-based 3D fuzzy modeling method is proposed. The agent improves the strategy by continuously interacting with the environment, so that the 3D fuzzy model can adaptively establish the online model from scratch. Specifically, this paper combines the deterministic strategy gradient reinforcement learning algorithm based on an actor critic framework with a 3D fuzzy system. The actor function and critic function are represented by two 3D fuzzy systems and the critic function and actor function are updated alternately. The critic function uses a TD (0) target and is updated via the semi-gradient method; the actor function is updated by using the chain derivation rule on the behavior value function and the actor function is the established DPS online model. Since DPS modeling is a continuous problem, this paper proposes a TD (0) target based on average reward, which can effectively realize online modeling. The suggested methodology is implemented on a three-zone rapid thermal chemical vapor deposition reactor system and the simulation results demonstrate the efficacy of the methodology.

Incremental Spatiotemporal Learning for Online Modeling of Distributed Parameter Systems

Online Spatiotemporal Modeling for High Spatial-Dimensional DPSs under Nonstationary Sensor Layout

Dual-Scale Learning-Based Online Modeling of Nonlinear Distributed Parameter Systems Under Time-Varying Boundary Conditions

Self-active and Recursively Selective Gaussian Process Models for Nonlinear Distributed Parameter Systems

Two-Dimensional Spatial Construction for Online Modeling of Distributed Parameter Systems

Reinforcement Learning-Based Optimal Sensor Placement for Spatiotemporal Modeling

Two-stage space construction for real-time modeling of distributed parameter systems under sparse sensing

Time/space-separation-based svm modeling for nonlinear distributed parameter processes

Probabilistic Pca Based Spatio-Temporal Multi-Modeling for Distributed Parameter Processes

Online Three-Dimensional Fuzzy Reinforcement Learning Modeling for Nonlinear Distributed Parameter Systems

Probabilistic PCA-Based Spatiotemporal Multimodeling for Nonlinear Distributed Parameter Processes

Time-space separation-based data driven method for monitoring distributed parameter process with sparse and noisy sensor data

Time–space Coupled Learning Method for Model Reduction of Distributed Parameter Systems with Encoder‐decoder and RNN

Greatly enhancing the modeling accuracy for distributed parameter systems by nonlinear time/space separation

Multi-Modeling for Nonlinear Distributed Parameter Processes Using Spatio-Temporal Kernel

Spatio-temporal learning with the online finite and infinite echo-state Gaussian processes

A Fuzzy-Based Spatio-Temporal Multi-Modeling for Nonlinear Distributed Parameter Processes

Spatiotemporal prediction for nonlinear parabolic distributed parameter system using an artificial neural network trained by group search optimization

Incremental Tensor Subspace Learning and Its Applications to Foreground Segmentation and Tracking

Online Joint State Inference and Learning of Partially Unknown State-Space Models

Nonlinear model predictive control for distributed parameter systems by time-space-coupled model reduction