Abstract:Variational auto-encoders (VAEs) have been widely used in soft sensing due to their ability to provide a probabilistic description of the hidden space. However, VAEs are static models that do not consider process dynamics, which can limit the ability of VAEs to accurately model complex industrial processes. To tackle this problem, this paper proposes a model called adaptive multi-head self-attention based supervised VAE (AMSA-SVAE). In AMSA-SVAE, an adaptive multi-head self-attention mechanism (AMSA) is proposed based on the multi-head self-attention mechanism (MSA). AMSA can dynamically extract different attention information depending on specific tasks. By adjusting the attention weights based on the input sequence, AMSA allows for more accurate and efficient modeling of complex industrial processes. Then, AMSA is used as the encoder and decoder of SVAE for soft sensing. Furthermore, with the data generation capabilities of VAE, an adaptive multi-head self-attention based VAE (AMSA-VAE) framework is proposed to address the issue of missing data. The AMSA-VAE is used to dynamically fill in missing data, thereby extending the capabilities of AMSA-SVAE. Finally, the performance of AMSA-SVAE is verified by a set of real industrial data, and the ability of AMSA-VAE framework is demonstrated by simulating different degrees of data missing rates. By combining the dynamic modeling capabilities of AMSA-SVAE with the data generation capabilities of AMSA-VAE, the proposed approach provides a robust solution to the challenges of incomplete data in soft sensing. Note to Practitioners— Soft sensors are widely used to measure key parameters in industrial processes, but missing values in the data are common due to sensor failures or transmission signal interference. This poses a significant challenge for traditional soft sensors, which require complete data to accurately model. Meanwhile, the dynamic nature of industrial process data further complicates the modeling process. To solve these challenges, this paper proposes an AMSA-SVAE model for soft sensing and an AMSA-VAE framework for filling in the missing values in the data, thereby extending the capabilities of AMSA-SVAE to handle missing data. When facing a dataset with missing values, AMSA-VAE framework is first used to fill in the missing values before the filled complete data is fed into AMSA-SVAE for modeling. Finally, the proposed approaches are evaluated through two sets of experiments using a real industrial dataset, showing the excellent performance of AMSA-SVAE and AMSA-VAE framework in modeling dynamic industrial process data and addressing the missing data problem.

Data Augmentation Using Time Conditional Variational Autoencoder for Soft Sensor of Industrial Processes With Limited Data

Semi-supervised Deep Conditional Variational Auto-encoder for Soft Sensor Modeling

Long-short Memory with Feature Representation by Latent-variable Model for Industrial Soft Sensor

Improving the Performance of Just-In-Time Learning-Based Soft Sensor Through Data Augmentation

Stacked Spatial–Temporal Autoencoder for Quality Prediction in Industrial Processes

Synthesizing labeled data to enhance soft sensor performance in data-scarce regions

Deep Learning of Process Data with Supervised Variational Auto-encoder for Soft Sensor

SVAE-WGAN-Based Soft Sensor Data Supplement Method for Process Industry

Novel virtual sample generation using conditional GAN for developing soft sensor with small data

Gaussian Mixture Model and Double-Weighted Deep Neural Networks for Data Augmentation Soft Sensing

Novel Virtual Sample Generation Using Target-Relevant Autoencoder for Small Data-Based Soft Sensor

A Novel Streaming Variational Bayesian Supervised Factor Analysis For Industrial Adaptive Soft Sensor Modeling

Predictive Modeling with Multiresolution Pyramid VAE and Industrial Soft Sensor Applications.

Adaptive Multi-Head Self-Attention Based supervised VAE for Industrial Soft Sensing With Missing Data

Supervised Multi-Layer Conditional Variational Auto-Encoder for Process Modeling and Soft Sensor

Supervised Nonlinear Dynamic System for Soft Sensor Application Aided by Variational Auto-Encoder

Novel virtual sample generation method based on data augmentation and weighted interpolation for soft sensing with small data

TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generation

Soft-Sensor Development Through Deep Learning with Spatial and Temporal Feature Extraction of Complex Processes

Novel Multimode Process Soft Sensing Methods Based on the Dynamic Mixture Variational Autoencoder Regression Model

Enhanced virtual sample generation based on manifold features: Applications to developing soft sensor using small data