Abstract:In recent times, Large Language Models (LLMs) have captured a global spotlight and revolutionized the field of Natural Language Processing. One of the factors attributed to the effectiveness of LLMs is the model architecture used for training, transformers. Transformer models excel at capturing contextual features in sequential data since time series data are sequential, transformer models can be leveraged for more efficient time series data prediction. The field of prognostics is vital to system health management and proper maintenance planning. A reliable estimation of the remaining useful life (RUL) of machines holds the potential for substantial cost savings. This includes avoiding abrupt machine failures, maximizing equipment usage, and serving as a decision support system (DSS). This work proposed an encoder-transformer architecture-based framework for multivariate time series prediction for a prognostics use case. We validated the effectiveness of the proposed framework on all four sets of the C-MAPPS benchmark dataset for the remaining useful life prediction task. To effectively transfer the knowledge and application of transformers from the natural language domain to time series, three model-specific experiments were conducted. Also, to enable the model awareness of the initial stages of the machine life and its degradation path, a novel expanding window method was proposed for the first time in this work, it was compared with the sliding window method, and it led to a large improvement in the performance of the encoder transformer model. Finally, the performance of the proposed encoder-transformer model was evaluated on the test dataset and compared with the results from 13 other state-of-the-art (SOTA) models in the literature and it outperformed them all with an average performance increase of 137.65% over the next best model across all the datasets.

An Ensemble of Transformer and LSTM Approach for Multivariate Time Series Data Classification

Time Series Classification using Deep Learning for Process Planning: A Case from the Process Industry

Multivariate Time Series Classification: A Deep Learning Approach

Anomaly detection in multivariate time series data using deep ensemble models

Time-Series Classification in Smart Manufacturing Systems: An Experimental Evaluation of State-of-the-Art Machine Learning Algorithms

TCLN: A Transformer-based Conv-LSTM Network for Multivariate Time Series Forecasting

Exploiting Multi-Channels Deep Convolutional Neural Networks for Multivariate Time Series Classification

From anomaly detection to classification with graph attention and transformer for multivariate time series

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification

Multiscale spatial‐temporal transformer with consistency representation learning for multivariate time series classification

Soft-Sensing ConFormer: A Curriculum Learning-based Convolutional Transformer

Soft Sensing Transformer: Hundreds of Sensors are Worth a Single Word

Mgformer: Multi-group transformer for multivariate time series classification

A two-stage adversarial Transformer based approach for multivariate industrial time series anomaly detection

A Transformer-based Framework For Multi-variate Time Series: A Remaining Useful Life Prediction Use Case

Densely Knowledge-Aware Network for Multivariate Time Series Classification

AutoLDT: a lightweight spatio-temporal decoupling transformer framework with AutoML method for time series classification

Multivariate Time Series Early Classification Across Channel and Time Dimensions

SVP-T: A Shape-Level Variable-Position Transformer for Multivariate Time Series Classification

MagNet: Multilevel Dynamic Wavelet Graph Neural Network for Multivariate Time Series Classification