Abstract:Nowadays, the deep learning technique has been widely applied in soft sensing benefiting from its strong ability of feature representation. However, classic dynamic soft sensing methods based on deep learning do not consider spatial-temporal and quality-relevant information simultaneously in the process data. Additionally, feature extraction methods encounter the challenge of information redundancy when dealing with large datasets. To address these challenges, we propose a novel hybrid self-attention mechanism with distributed GRUs (HSAM-dGRUs) within an encoder-decoder framework. The HSAM-dGRUs architecture contains a hybrid self-attention encoder, integrating multi-channel multi-head self-attention (MC-MSA) and quality-related self-attention (QR-SAM). The MC-MSA method effectively extracts local features through multi-channels to capture spatio-temporal characteristics in sequence data. Subsequently, the QR-SAM method introduces supervisory knowledge into the features and capture the quality-relevant information adaptively. The distributed GRUs decoder is designed to extract local dynamic hidden states from multi-channel features for accurate prediction. Two case studies on real industrial process datasets demonstrate the effectiveness and superiority of the proposed HSAM-dGRUs soft sensing approach. Note to Practitioners—In the process industry, monitoring quality variables is beneficial for tracking process status, saving energy, and reducing emissions. However, it is difficult to implement accurate measurement and real-time control for process quality in general. In this work, we propose a novel hybrid self-attention mechanism utilizing distributed gated recurrent units for soft sensing modeling of dynamic processes. The proposed method integrates an multi-channel multi-head self-attention mechanism and a quality self-attention mechanism into the encoder. The encoder is responsible for adaptively extracting spatio-temporal features and assigning weights based on quality-related information. Furthermore, a decoder based on distributed GRUs is employed to capture the dynamic information from the extracted features, enhancing the prediction task. First, the proposed methods can be trained offline based on historical dataset. Then, the well-trained model can be uploaded and use the new data for accurate online prediction. Preliminary experiments in this paper have demonstrated the feasibility of the proposed method, but it has not been tested in actual industrial production. In the future, we will work on developing a quality prediction system based on the proposed method so that it can be applied in actual industrial production.

Attention-Based Interval Aided Networks for Data Modeling of Heterogeneous Sampling Sequences with Missing Values in Process Industry

Quality Prediction Model for Process Sequential Data of Irregular Measurements with Sampling-Interval-attention LSTM

A SIA-LSTM Based Virtual Metrology for Quality Variables in Irregular Sampled Time Sequence of Industrial Processes

Hierarchical Self-Attention Network for Industrial Data Series Modeling with Different Sampling Rates Between the Input and Output Sequences

Nonlinear semi-parametric modeling method based on GA-ANN

Deep learning for quality prediction of nonlinear dynamic processes with variable attention-based long short-term memory network

Missing Data Imputation for Industrial Time Series with Adaptive Median Iteration Based on Generative Adversarial Networks

A Spatiotemporal Industrial Soft Sensor Modeling Scheme for Quality Prediction With Missing Data

Deep Neural Network-Embedded Stochastic Nonlinear State-Space Models and Their Applications to Process Monitoring

AMING:Adaptive-Learning Mechanism Based Missing-Value Imputation Networks Using GAN in Industrial Processes

Real-time Estimation of Quality-Related Variable for Dynamic and Non-Gaussian Process Based on Semisupervised Bayesian HMM

A Novel Incremental Gaussian Mixture Regression and Its Application for Time-varying Multimodal Process Quality Prediction

A Multi-Step Sequence-to-Sequence Model with Attention LSTM Neural Networks for Industrial Soft Sensor Application

Dynamic Historical Information Incorporated Attention Deep Learning Model for Industrial Soft Sensor Modeling

Deep Learning with Spatiotemporal Attention-Based LSTM for Industrial Soft Sensor Model Development

Multi-feature generation network-based imputation method for industrial data with high missing rate

Frame-Dilated Convolutional Fusion Network and GRU-Based Self-Attention Dual-Channel Network for Soft-Sensor Modeling of Industrial Process Quality Indexes

Multi-modal Hybrid Modeling Strategy Based on Gaussian Mixture Variational Autoencoder and Spatial–temporal Attention: Application to Industrial Process Prediction

Interval-Aware Probabilistic Slow Feature Analysis for Irregular Dynamic Process Monitoring with Missing Data

Semi-supervised LSTM with Historical Feature Fusion Attention for Temporal Sequence Dynamic Modeling in Industrial Processes

Novel Distributed GRUs Based on Hybrid Self-Attention Mechanism for Dynamic Soft Sensing