Abstract:Real-time monitoring and accurate prediction of key variables are indispensable to ensure industrial production activities proceed as expected. With the increase in measurement data volume and the improvement of hardware computing power, the Transformer and its variants, due to their excellent capability in extracting global dependencies, are playing an increasingly important role among deep learning-based multidimensional time series prediction models. In addition, from the perspective of causality, cause variables contain parts of information in effect variables and can reduce the uncertainty of effect variables, which is beneficial for prediction. However, there has been relatively limited research on combining the Transformer and causal feature analysis. To fully use both advantages, this paper introduces the Causal-Transformer (CT) model, which utilizes semi-orthogonal projection to extract causal features from multiple input variables. A multi-head spatial-temporal causal attention mechanism is designed in the encoder block based on the classical Transformer model to simultaneously reduce feature dimensions and extract implicit causal features in both the temporal and spatial dimensions. The CT also utilizes the Granger causality analysis to select the causal teaching indicators of target variables to provide stable assistance by injecting explicit causality into the inputs of the decoder block. By leveraging more condensed and independent causal features, the CT possesses inherent advantages in predicting time series variables. Case study results show that the CT model outperforms the other models on the diesel refinery dataset, especially with a reduction of 46.0% and 30.4% in MSE towards the classic Transformer and informer in five-step prediction. Copyright (C)2024 The Authors. This is an open access article under the CC BY-NC-ND license (htips://creativecommons.org/licenses/by-nc-nd/4.0/)

Large-scale Chemical Process Causal Discovery from Big Data with Transformer-Based Deep Learning

Spatial-Temporal Causality Modeling for Industrial Processes with a Knowledge-Data Guided Reinforcement Learning

Multichannel Dynamic Graph Convolutional Network-Based Fault Diagnosis and Its Application in Blast Furnace Ironmaking Process

A Novel Transformer-Based Model with Large Kernel Temporal Convolution for Chemical Process Fault Detection

A novel deep learning model based on target transformer for fault diagnosis of chemical process

Root Cause and Fault Propagation Analysis Based on Causal Graph in Chemical Processes

A Novel Fault Detection and Identification Method for Complex Chemical Processes Based on OSCAE and CNN

Causality-embedded Reconstruction Network for High-Resolution Fault Identification in Chemical Process

A novel transformer-based multi-variable multi-step prediction method for chemical process fault prognosis

Generalized Transformer in Fault Diagnosis of Tennessee Eastman Process

A Novel Label-Aware Global Graph Construction Method and Spiking-Coded Graph Neural Network for Intelligent Process Fault Diagnosis

Root cause diagnosis for complex industrial process faults via spatiotemporal coalescent based time series prediction and optimized Granger causality

Causal inference of multivariate time series in complex industrial systems

Causal-Transformer: Spatial-temporal Causal Attention-Based Transformer for Time Series Prediction

Causal Variable Selection for Industrial Process Quality Prediction Via Attention-Based GRU Network

Research on Fault Diagnosis Method Based on Structural Causal Model in Tennessee Eastman Process

A Process Data Prediction Method for Chemical Process Based on the Frozen Pretrained Transformer Model

Data-driven Causal Inference Based on a Modified Transfer Entropy

Fault Diagnosis of the Dynamic Chemical Process Based on the Optimized CNN-LSTM Network

Causal Generative Model for Root-Cause Diagnosis and Fault Propagation Analysis in Industrial Processes

DSTN: Dynamic Spatio-Temporal Network for Early Fault Warning in Chemical Processes