Sparse Low-Ranked Self-Attention Transformer for Remaining Useful Lifetime Prediction of Optical Fiber Amplifiers

Dominic Schneider,Lutz Rapp

2024-09-27

Abstract:Optical fiber amplifiers are key elements in present optical networks. Failures of these components result in high financial loss of income of the network operator as the communication traffic over an affected link is interrupted. Applying Remaining useful lifetime (RUL) prediction in the context of Predictive Maintenance (PdM) to optical fiber amplifiers to predict upcoming system failures at an early stage, so that network outages can be minimized through planning of targeted maintenance actions, ensures reliability and safety. Optical fiber amplifier are complex systems, that work under various operating conditions, which makes correct forecasting a difficult task. Increased monitoring capabilities of systems results in datasets that facilitate the application of data-driven RUL prediction methods. Deep learning models in particular have shown good performance, but generalization based on comparatively small datasets for RUL prediction is difficult. In this paper, we propose Sparse Low-ranked self-Attention Transformer (SLAT) as a novel RUL prediction method. SLAT is based on an encoder-decoder architecture, wherein two parallel working encoders extract features for sensors and time steps. By utilizing the self-attention mechanism, long-term dependencies can be learned from long sequences. The implementation of sparsity in the attention matrix and a low-rank parametrization reduce overfitting and increase generalization. Experimental application to optical fiber amplifiers exemplified on EDFA, as well as a reference dataset from turbofan engines, shows that SLAT outperforms the state-of-the-art methods.

Machine Learning,Artificial Intelligence,Signal Processing

What problem does this paper attempt to address?

The paper attempts to address the problem of how to identify system failures in advance through Remaining Useful Lifetime (RUL) prediction in Predictive Maintenance (PdM) of Optical Fiber Amplifiers, thereby reducing network downtime and improving network reliability. Specifically, the paper proposes a new Transformer-based method—Sparse Low-ranked self-Attention Transformer (SLAT)—to tackle the challenges of modeling the complexity and long-term dependencies of optical amplifiers under different operating conditions, aiming to enhance generalization capability on small datasets for more accurate RUL prediction. The core issues of the paper can be summarized as: 1. How to effectively extract features from monitoring data for RUL prediction. 2. How to handle long-term dependencies in long sequence data, especially in the case of small datasets. 3. How to reduce overfitting and improve the model's generalization ability by introducing sparsity and low-rank parameterization. By addressing these issues, the paper hopes to achieve earlier and more accurate fault prediction in the predictive maintenance of optical amplifiers, thereby optimizing maintenance plans and reducing economic losses and network interruptions caused by equipment failures.

Sparse Low-Ranked Self-Attention Transformer for Remaining Useful Lifetime Prediction of Optical Fiber Amplifiers

Aero-Engine Remaining Useful Life Prediction Via Physics-Informed Self-Attention Encoder

A Novel Deep Learning-Based Encoder-Decoder Model for Remaining Useful Life Prediction

Remaining useful life prediction based on spatiotemporal autoencoder

Efficient temporal flow Transformer accompanied with multi-head probsparse self-attention mechanism for remaining useful life prognostics

An integrated multi-head dual sparse self-attention network for remaining useful life prediction

Trend-augmented and temporal-featured Transformer network with multi-sensor signals for remaining useful life prediction

Structural-Guided Interaction and Attention-Enhancing Compensation Network for Machine Remaining Useful Life Prediction

An attention-based temporal convolutional network method for predicting remaining useful life of aero-engine

A novel transformer-based DL model enhanced by position-sensitive attention and gated hierarchical LSTM for aero-engine RUL prediction

Multiscale Feature Extension Enhanced Deep Global-Local Attention Network for Remaining Useful Life Prediction

A Bayesian Adversarial Probsparse Transformer model for long-term RUL prediction

Remaining Useful Life Prediction for Aircraft Engines under High-Pressure Compressor Degradation Faults Based on FC-AMSLSTM

A dual attention LSTM lightweight model based on exponential smoothing for remaining useful life prediction

DRLSTM-DA: a novel aeroengine remaining useful life prediction method based on deep optimization of multiple sensors information

Bearing Remaining Useful Life Prediction Using Federated Learning With Taylor-Expansion Network Pruning

An Improved Method for Predicting the Remaining Useful Life Using a Spatial–Temporal Feature Extraction Network With Attention Mechanism

Channel attention & temporal attention based temporal convolutional network: A dual attention framework for remaining useful life prediction of the aircraft engines

Remaining Useful Life Prediction: A Study on Multidimensional Industrial Signal Processing and Efficient Transfer Learning Based on Large Language Models

Multi-Head Self-Attention-Based Fully Convolutional Network for RUL Prediction of Turbofan Engines

A concise self-adapting deep learning network for machine remaining useful life prediction