Abstract:Accurate imputation of missing data is pivotal in real-world industrial applications. Traditional direct imputers, which utilize basic statistics to replace missing elements, offer a practical solution but struggle to adapt to the complex patterns in industrial data, leaving a gap in the research landscape. This study explores the untapped potential of direct imputers, enhancing their adaptability and capacity to handle complex patterns in industrial data through optimal transport (OT) theory, with a focus on preserving local sample-wise similarity as an exemplar. To these ends, we construct a Local Similarity Preserved Transport (LSPT) problem, with a solution algorithm based on the Frank-Wolfe technique to compute transport cost. Subsequently, we propose the LSPT-D framework, which employs the transport cost of LSPT for distribution matching, directing the gradient flow to the missing data points to update the imputations directly. This strategy maintains local similarity throughout the imputation process thereby enhancing the overall imputation quality. Our experiments demonstrate that LSPT-D outperform various baselines in industrial missing data imputation. Note to Practitioners —Accurate missing data imputation is essential for enhancing the reliability of data analytics and reducing decision-making risks in industrial automation. This study introduces LSPT-D, a non-parametric imputation technique based on OT technology. Unique to LSPT-D is its ability to preserve local similarity during the imputation process, rendering it particularly advantageous for datasets with varying operational phases and load conditions. In industrial applications, LSPT-D not only significantly improves imputation quality compared to various baseline methods but also maintains modest running costs. Additionally, it serves as an exemplar for developing OT-based imputation strategies that capitalize on the inherent properties of data to improve imputation performance. However, LSPT-D operates under the independent and identically distributed assumption and is thus best applied in scenarios where temporal dependencies, such as trends and seasonality, are minimal or have been previously neutralized.

Missing-Data Imputation with Position-Encoding Denoising Auto-encoders for Industrial Processes

Exploiting SDAE Model for Recommendations

Multiple Imputation with Denoising Autoencoder using Metamorphic Truth and Imputation Feedback

Stacked Spatial–Temporal Autoencoder for Quality Prediction in Industrial Processes

Denoising Diffusion Straightforward Models for Energy Conversion Monitoring Data Imputation

LSPT-D: Local Similarity Preserved Transport for Direct Industrial Data Imputation

mDAE : modified Denoising AutoEncoder for missing data imputation

MIDIA: exploring denoising autoencoders for missing data imputation

Smoothed LSTM-AE: A spatio-temporal deep model for multiple time-series missing imputation

A transferred spatio-temporal deep model based on multi-LSTM auto-encoder for air pollution time series missing value imputation

Features Masked Auto-Encoder-Based Anomaly Detection in Process Industry

Robust Monitoring and Fault Isolation of Nonlinear Industrial Processes Using Denoising Autoencoder and Elastic Net

Siamese autoencoder architecture for the imputation of data missing not at random

Multiscale-attention Masked Autoencoder for Missing Data Imputation of Wind Turbines

Multi-feature generation network-based imputation method for industrial data with high missing rate

Opening the Black Box: Towards inherently interpretable energy data imputation models using building physics insight

Stacked Denoising Autoencoders Based Poisson Regression For Count Data Modeling

Proposition of a Theoretical Model for Missing Data Imputation using Deep Learning and Evolutionary Algorithms

Gated Stacked Target-Related Autoencoder: A Novel Deep Feature Extraction and Layerwise Ensemble Method for Industrial Soft Sensor Application

Deep Learning for Industrial KPI Prediction: when Ensemble Learning Meets Semi-Supervised Data