Abstract:Variational auto-encoders (VAEs) have been widely used in soft sensing due to their ability to provide a probabilistic description of the hidden space. However, VAEs are static models that do not consider process dynamics, which can limit the ability of VAEs to accurately model complex industrial processes. To tackle this problem, this paper proposes a model called adaptive multi-head self-attention based supervised VAE (AMSA-SVAE). In AMSA-SVAE, an adaptive multi-head self-attention mechanism (AMSA) is proposed based on the multi-head self-attention mechanism (MSA). AMSA can dynamically extract different attention information depending on specific tasks. By adjusting the attention weights based on the input sequence, AMSA allows for more accurate and efficient modeling of complex industrial processes. Then, AMSA is used as the encoder and decoder of SVAE for soft sensing. Furthermore, with the data generation capabilities of VAE, an adaptive multi-head self-attention based VAE (AMSA-VAE) framework is proposed to address the issue of missing data. The AMSA-VAE is used to dynamically fill in missing data, thereby extending the capabilities of AMSA-SVAE. Finally, the performance of AMSA-SVAE is verified by a set of real industrial data, and the ability of AMSA-VAE framework is demonstrated by simulating different degrees of data missing rates. By combining the dynamic modeling capabilities of AMSA-SVAE with the data generation capabilities of AMSA-VAE, the proposed approach provides a robust solution to the challenges of incomplete data in soft sensing. Note to Practitioners— Soft sensors are widely used to measure key parameters in industrial processes, but missing values in the data are common due to sensor failures or transmission signal interference. This poses a significant challenge for traditional soft sensors, which require complete data to accurately model. Meanwhile, the dynamic nature of industrial process data further complicates the modeling process. To solve these challenges, this paper proposes an AMSA-SVAE model for soft sensing and an AMSA-VAE framework for filling in the missing values in the data, thereby extending the capabilities of AMSA-SVAE to handle missing data. When facing a dataset with missing values, AMSA-VAE framework is first used to fill in the missing values before the filled complete data is fed into AMSA-SVAE for modeling. Finally, the proposed approaches are evaluated through two sets of experiments using a real industrial dataset, showing the excellent performance of AMSA-SVAE and AMSA-VAE framework in modeling dynamic industrial process data and addressing the missing data problem.

Semi-supervised Deep Conditional Variational Auto-encoder for Soft Sensor Modeling

Deep Learning of Process Data with Supervised Variational Auto-encoder for Soft Sensor

Supervised Multi-Layer Conditional Variational Auto-Encoder for Process Modeling and Soft Sensor

Long-short Memory with Feature Representation by Latent-variable Model for Industrial Soft Sensor

Supervised Nonlinear Dynamic System for Soft Sensor Application Aided by Variational Auto-Encoder

Semi-Supervised Deep Dynamic Probabilistic Latent Variable Model for Multimode Process Soft Sensor Application

Novel Multimode Process Soft Sensing Methods Based on the Dynamic Mixture Variational Autoencoder Regression Model

A Model Combining Seq2Seq Network and LightGBM Algorithm for Industrial Soft Sensor

Weighted Nonlinear Dynamic System for Deep Extraction of Nonlinear Dynamic Latent Variables and Industrial Application

Data Augmentation Using Time Conditional Variational Autoencoder for Soft Sensor of Industrial Processes With Limited Data

Predictive Modeling with Multiresolution Pyramid VAE and Industrial Soft Sensor Applications.

Nonlinear Probabilistic Latent Variable Regression Models for Soft Sensor Application: from Shallow to Deep Structure

Adaptive Multi-Head Self-Attention Based supervised VAE for Industrial Soft Sensing With Missing Data

Deep Learning of Semisupervised Process Data with Hierarchical Extreme Learning Machine and Soft Sensor Application

Synthesizing labeled data to enhance soft sensor performance in data-scarce regions

Dynamic Features Incorporated Locally Weighted Deep Learning Model for Soft Sensor Development.

Information Complementary Fusion Stacked Autoencoders for Soft Sensor Applications in Multimode Industrial Processes

Gated Stacked Target-Related Autoencoder: A Novel Deep Feature Extraction and Layerwise Ensemble Method for Industrial Soft Sensor Application

Semi-supervised ensemble support vector regression based soft sensor for key quality variable estimation of nonlinear industrial processes with limited labeled data

Deep Learning for Industrial KPI Prediction: when Ensemble Learning Meets Semi-Supervised Data

Soft-Sensor Development Through Deep Learning with Spatial and Temporal Feature Extraction of Complex Processes