Abstract:In recent times, Large Language Models (LLMs) have captured a global spotlight and revolutionized the field of Natural Language Processing. One of the factors attributed to the effectiveness of LLMs is the model architecture used for training, transformers. Transformer models excel at capturing contextual features in sequential data since time series data are sequential, transformer models can be leveraged for more efficient time series data prediction. The field of prognostics is vital to system health management and proper maintenance planning. A reliable estimation of the remaining useful life (RUL) of machines holds the potential for substantial cost savings. This includes avoiding abrupt machine failures, maximizing equipment usage, and serving as a decision support system (DSS). This work proposed an encoder-transformer architecture-based framework for multivariate time series prediction for a prognostics use case. We validated the effectiveness of the proposed framework on all four sets of the C-MAPPS benchmark dataset for the remaining useful life prediction task. To effectively transfer the knowledge and application of transformers from the natural language domain to time series, three model-specific experiments were conducted. Also, to enable the model awareness of the initial stages of the machine life and its degradation path, a novel expanding window method was proposed for the first time in this work, it was compared with the sliding window method, and it led to a large improvement in the performance of the encoder transformer model. Finally, the performance of the proposed encoder-transformer model was evaluated on the test dataset and compared with the results from 13 other state-of-the-art (SOTA) models in the literature and it outperformed them all with an average performance increase of 137.65% over the next best model across all the datasets.

Time-series forecasting of mortality rates using transformer

Foreformer: an Enhanced Transformer-Based Framework for Multivariate Time Series Forecasting

Hidformer: Hierarchical Dual-Tower Transformer Using Multi-Scale Mergence for Long-Term Time Series Forecasting

Enhancing Time Series Forecasting: A Hierarchical Transformer with Probabilistic Decomposition Representation

Short-Term Speed Forecasting of Large-Scale Urban Road Network Based on Transformer

Transformers for cardiac patient mortality risk prediction from heterogeneous electronic health records

A Deep Learning Approach to the Transformer Life Prediction Considering Diverse Aging Factors

Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformers

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

Dateformer: Time-modeling Transformer for Longer-term Series Forecasting

Advancing mortality rate prediction in European population clusters: integrating deep learning and multiscale analysis

Advanced Risk Prediction and Stability Assessment of Banks Using Time Series Transformer Models

A Transformer-based Framework For Multi-variate Time Series: A Remaining Useful Life Prediction Use Case

Two Steps Forward and One Behind: Rethinking Time Series Forecasting with Deep Learning

Long-term prediction for temporal propagation of seasonal influenza using Transformer-based model

Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case

Transformer-based power system energy prediction model

Itransformer: Inverted Transformers Are Effective for Time Series Forecasting

Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting

Prediction of Post-Operative Renal and Pulmonary Complications Using Transformers

NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting