Abstract:In recent times, Large Language Models (LLMs) have captured a global spotlight and revolutionized the field of Natural Language Processing. One of the factors attributed to the effectiveness of LLMs is the model architecture used for training, transformers. Transformer models excel at capturing contextual features in sequential data since time series data are sequential, transformer models can be leveraged for more efficient time series data prediction. The field of prognostics is vital to system health management and proper maintenance planning. A reliable estimation of the remaining useful life (RUL) of machines holds the potential for substantial cost savings. This includes avoiding abrupt machine failures, maximizing equipment usage, and serving as a decision support system (DSS). This work proposed an encoder-transformer architecture-based framework for multivariate time series prediction for a prognostics use case. We validated the effectiveness of the proposed framework on all four sets of the C-MAPPS benchmark dataset for the remaining useful life prediction task. To effectively transfer the knowledge and application of transformers from the natural language domain to time series, three model-specific experiments were conducted. Also, to enable the model awareness of the initial stages of the machine life and its degradation path, a novel expanding window method was proposed for the first time in this work, it was compared with the sliding window method, and it led to a large improvement in the performance of the encoder transformer model. Finally, the performance of the proposed encoder-transformer model was evaluated on the test dataset and compared with the results from 13 other state-of-the-art (SOTA) models in the literature and it outperformed them all with an average performance increase of 137.65% over the next best model across all the datasets.

PAOLTransformer: Pruning-adaptive Optimal Lightweight Transformer Model for Aero-Engine Remaining Useful Life Prediction.

Transformer Model for Remaining Useful Life Prediction of Aeroengine

Aero-Engine Remaining Useful Life Prediction Via Physics-Informed Self-Attention Encoder

Remaining Useful Life Prediction of Aero-Engine Based on Transformer with Tendency Retainment

An Aircraft Engine Remaining Useful Life Prediction Method Based on Predictive Vector Angle Minimization and Feature Fusion Gate Improved Transformer Model

A Two-Stage Attention-Based Hierarchical Transformer for Turbofan Engine Remaining Useful Life Prediction

A lightweight transformer and depthwise separable convolution model for remaining useful life prediction of turbofan engines

A novel transformer-based DL model enhanced by position-sensitive attention and gated hierarchical LSTM for aero-engine RUL prediction

Using transformer and a reweighting technique to develop a remaining useful life estimation method for turbofan engines

A Bayesian Adversarial Probsparse Transformer model for long-term RUL prediction

Domain Adaptive Remaining Useful Life Prediction with Transformer.

A Transformer-based Framework For Multi-variate Time Series: A Remaining Useful Life Prediction Use Case

A new dual-channel Transformer-based Network for Remaining Useful Life Prediction

A Hybrid Transformer-GRU Network for Remain Useful Life Prediction of Aero-Engine Systems

End-to-end Temporal Transformer with Autocorrelated Attention Mechanism Augmented for Remaining Useful Life Prediction of Turbopump Bearings

Exhaust Gas Temperature Prediction of Aero-Engine via Enhanced Scale-Aware Efficient Transformer

Aero-Engine Remaining Useful Life Prediction Via Tensor Decomposition Method

Transformer Encoder Enhanced by an Adaptive Graph Convolutional Neural Network for Prediction of Aero-Engines' Remaining Useful Life

Aero-Engine Remaining Useful Life Estimation Based on CAE-TCN Neural Networks

Prediction of Remaining Useful Life Using Fused Deep Learning Models: A Case Study of Turbofan Engines

Remaining Useful Life Prediction for Aero-Engines Using a Time-Enhanced Multi-Head Self-Attention Model