Variational Inference over Graph: Knowledge Representation for Deep Process Data Analytics

Zhichao Chen,Zhihuan Song,Zhiqiang Ge
DOI: https://doi.org/10.1109/tkde.2023.3327415
IF: 9.235
2024-01-01
IEEE Transactions on Knowledge and Data Engineering
Abstract:With the advent of the industrial Big Data era, accurate estimation of product quality and monitoring of working conditions from historical data have become crucial in the process industry. However, the majority of data-driven approaches predominantly rely on observational data, overlooking the valuable empirical knowledge derived from experience or underlying mechanisms. In order to leverage this knowledge, researchers employ variousgraphneuralnetwork-basedmethodswhichintroduceconnectionsamongprocess variables for feature extraction. Nevertheless, it is imperative to recognize that process knowledge undergoes changes due to internal or external concept drift. To address this challenge, we propose a novel deep learning module called “variational inference over graph” to effectively harness shifting knowledge. Building upon the self-attention mechanism, we design a probabilistic self-attention mechanism for encoding and reconciling prior knowledge. Instead of directly encoding the prior knowledge through graph neural network edges, we incorporate it as regularization term within the variational inference framework that accounts for knowledge shift. Furthermore, we introduce reparameterization estimator to control the variance resulting from knowledge uncertainty. To showcase the capability of our proposed method, we conduct various experiments on quality prediction tasks in real industrial process.
What problem does this paper attempt to address?