Abstract:With the rapid deployment of cloud platforms, high service reliability is of critical importance. An industrial cloud platform contains a huge number of disks, and disk failure is a common cause of service unreliability. In recent years, many machine learning based disk failure prediction approaches have been proposed, and they can predict disk failures based on disk status data before the failures actually happen. In this way, proactive actions can be taken in advance to improve service reliability. However, existing approaches treat each disk individually and do not explore the influence of the neighboring disks. In this paper, we propose Neighborhood-Temporal Attention Model (NTAM), a novel deep learning based approach to disk failure prediction. When predicting whether or not a disk will fail in near future, NTAM is a novel approach that not only utilizes a disk's own status data, but also considers its neighbors' status data. Moreover, NTAM includes a novel attention-based temporal component to capture the temporal nature of the disk status data. Besides, we propose a data enhancement method, called Temporal Progressive Sampling (TPS), to handle the extreme data imbalance issue. We evaluate NTAM on a public dataset as well as two industrial datasets collected from millions of disks in Microsoft Azure. Our experimental results show that NTAM significantly outperforms state-of-the-art competitors. Also, our empirical evaluations indicate the effectiveness of the neighborhood-ware component and the temporal component underlying NTAM as well as the effectiveness of TPS. More encouragingly, we have successfully applied NTAM and TPS to Microsoft cloud platforms (including Microsoft Azure and Microsoft 365) and obtained benefits in industrial practice.

Adversarial Domain Adaptation with Correlation-Based Association Networks for Longitudinal Disk Fault Prediction

Disk failure prediction based on multi-layer domain adaptive learning

A Stable Adaptive Adversarial Network with Exponential Adversarial Strategy for Bearing Fault Diagnosis

A Hybrid Adversarial Domain Adaptation Network for Bearing Fault Diagnosis Under Varying Working Conditions

Disk Failure Prediction in Data Centers Via Online Learning

Intelligent Fault Diagnosis With Deep Adversarial Domain Adaptation

A Failure Prediction Approach Based on BiLSTM and Deep Feature Extractor for Hard Disk Drives.

A Dual-View Alignment-Based Domain Adaptation Network for Fault Diagnosis

<i>NTAM</i>: Neighborhood-Temporal Attention Model for Disk Failure Prediction in Cloud Platforms

Bearing Fault Diagnosis Based on Deep Discriminative Adversarial Domain Adaptation Neural Networks

A Mixed Adversarial Adaptation Network for Intelligent Fault Diagnosis

A New Probability Guided Domain Adversarial Network for Bearing Fault Diagnosis

Unsupervised Adversarial Adaptation Network for Intelligent Fault Diagnosis

A Novel Bayesian Deep Dual Network with Unsupervised Domain Adaptation for Transfer Fault Prognosis Across Different Machines

Correlation Regularized Conditional Adversarial Adaptation for Multi-Target-Domain Fault Diagnosis

Double-level Adversarial Domain Adaptation Network for Intelligent Fault Diagnosis.

Tree-Based Model with Advanced Data Preprocessing for Large Scale Hard Disk Failure Prediction

Bearing fault diagnosis based on partial domain adaptation adversarial network

Open Set Bearing Fault Diagnosis with Domain Adaptive Adversarial Network under Varying Conditions

Dynamic Balanced Domain-Adversarial Networks for Cross-Domain Fault Diagnosis of Train Bearings

A locally weighted multi-domain collaborative adaptation for failure prediction in SSDs