Abstract:Variational auto-encoders (VAEs) have been widely used in soft sensing due to their ability to provide a probabilistic description of the hidden space. However, VAEs are static models that do not consider process dynamics, which can limit the ability of VAEs to accurately model complex industrial processes. To tackle this problem, this paper proposes a model called adaptive multi-head self-attention based supervised VAE (AMSA-SVAE). In AMSA-SVAE, an adaptive multi-head self-attention mechanism (AMSA) is proposed based on the multi-head self-attention mechanism (MSA). AMSA can dynamically extract different attention information depending on specific tasks. By adjusting the attention weights based on the input sequence, AMSA allows for more accurate and efficient modeling of complex industrial processes. Then, AMSA is used as the encoder and decoder of SVAE for soft sensing. Furthermore, with the data generation capabilities of VAE, an adaptive multi-head self-attention based VAE (AMSA-VAE) framework is proposed to address the issue of missing data. The AMSA-VAE is used to dynamically fill in missing data, thereby extending the capabilities of AMSA-SVAE. Finally, the performance of AMSA-SVAE is verified by a set of real industrial data, and the ability of AMSA-VAE framework is demonstrated by simulating different degrees of data missing rates. By combining the dynamic modeling capabilities of AMSA-SVAE with the data generation capabilities of AMSA-VAE, the proposed approach provides a robust solution to the challenges of incomplete data in soft sensing. Note to Practitioners— Soft sensors are widely used to measure key parameters in industrial processes, but missing values in the data are common due to sensor failures or transmission signal interference. This poses a significant challenge for traditional soft sensors, which require complete data to accurately model. Meanwhile, the dynamic nature of industrial process data further complicates the modeling process. To solve these challenges, this paper proposes an AMSA-SVAE model for soft sensing and an AMSA-VAE framework for filling in the missing values in the data, thereby extending the capabilities of AMSA-SVAE to handle missing data. When facing a dataset with missing values, AMSA-VAE framework is first used to fill in the missing values before the filled complete data is fed into AMSA-SVAE for modeling. Finally, the proposed approaches are evaluated through two sets of experiments using a real industrial dataset, showing the excellent performance of AMSA-SVAE and AMSA-VAE framework in modeling dynamic industrial process data and addressing the missing data problem.

Adaptive Multi-Head Self-Attention Based supervised VAE for Industrial Soft Sensing With Missing Data

Semi-supervised Deep Conditional Variational Auto-encoder for Soft Sensor Modeling

Long-short Memory with Feature Representation by Latent-variable Model for Industrial Soft Sensor

Predictive Modeling with Multiresolution Pyramid VAE and Industrial Soft Sensor Applications.

Stacked Spatial–Temporal Autoencoder for Quality Prediction in Industrial Processes

Supervised Nonlinear Dynamic System for Soft Sensor Application Aided by Variational Auto-Encoder

Novel Multimode Process Soft Sensing Methods Based on the Dynamic Mixture Variational Autoencoder Regression Model

Deep Learning of Process Data with Supervised Variational Auto-encoder for Soft Sensor

Data Augmentation Using Time Conditional Variational Autoencoder for Soft Sensor of Industrial Processes With Limited Data

M 2 D-VAE: Self-Supervised Probabilistic Temporal- Spatial Latent Representation Learning for Unsupervised Industrial Operational Applications Under Missing Value Interference

Synthesizing labeled data to enhance soft sensor performance in data-scarce regions

A Novel Streaming Variational Bayesian Supervised Factor Analysis For Industrial Adaptive Soft Sensor Modeling

Semi-Supervised Deep Dynamic Probabilistic Latent Variable Model for Multimode Process Soft Sensor Application

Streaming Parallel Variational Bayesian Supervised Factor Analysis for Adaptive Soft Sensor Modeling with Big Process Data

SVAE-WGAN-Based Soft Sensor Data Supplement Method for Process Industry

Supervised Multi-Layer Conditional Variational Auto-Encoder for Process Modeling and Soft Sensor

Mode Information Separated Β-Vae Regression for Multimode Industrial Process Soft Sensing

A Multi-Step Sequence-to-Sequence Model with Attention LSTM Neural Networks for Industrial Soft Sensor Application

Information Complementary Fusion Stacked Autoencoders for Soft Sensor Applications in Multimode Industrial Processes

Gated Stacked Target-Related Autoencoder: A Novel Deep Feature Extraction and Layerwise Ensemble Method for Industrial Soft Sensor Application

Variational Adversarial Active Learning Assisted Process Soft Sensor Method