Abstract:With the development of cloud computing and big data, the reliability of data storage systems becomes increasingly important. Previous researchers have shown that machine learning algorithms based on SMART attributes are effective methods to predict hard drive failures. In this paper, we use SMART attributes to predict hard drive health degrees which are helpful for taking different fault tolerant actions in advance. Given the highly imbalanced SMART datasets, it is a nontrivial work to predict the health degree precisely. The proposed model would encounter overfitting and biased fitting problems if it is trained by the traditional methods. In order to resolve this problem, we propose two strategies to better utilize imbalanced data and improve performance. Firstly, we design a layerwise perturbation-based adversarial training method which can add perturbations to any layers of a neural network to improve the generalization of the network. Secondly, we extend the training method to the semi-supervised settings. Then, it is possible to utilize unlabeled data that have a potential of failure to further improve the performance of the model. Our extensive experiments on two real-world hard drive datasets demonstrate the superiority of the proposed schemes for both supervised and semi-supervised classification. The model trained by the proposed method can correctly predict the hard drive health status 5 and 15 days in advance. Finally, we verify the generality of the proposed training method in other similar anomaly detection tasks where the dataset is imbalanced. The results argue that the proposed methods are applicable to other domains.

A Combined Bayesian Network Method for Predicting Drive Failure Times from SMART Attributes.

Proactive drive failure prediction for large scale storage systems

Hard Drive Failure Prediction Using Classification and Regression Trees

Health Status Assessment and Failure Prediction for Hard Drives with Recurrent Neural Networks

A Naive Bayesian Belief Network Model for Predicting Effort Deviation Rate in Software Testing

A Failure Prediction Approach Based on BiLSTM and Deep Feature Extractor for Hard Disk Drives.

Hard drive failure prediction using Decision Trees

Health Status Assessment for HDDs Based on Bi-LSTM and N-Dimensional Similarity Metric

Predicting Disk Failures with HMM- and HSMM-based Approaches

Hard Disk Failure Prediction Based on Lightgbm with CID.

Classification Based Hard Disk Drive Failure Prediction: Methodologies, Performance Evaluation and Comparison.

Predicting Hard Drive Failures for Cloud Storage Systems

Layerwise Perturbation-Based Adversarial Training for Hard Drive Health Degree Prediction

Hard Drive Failure Prediction Using Big Data

Hard Disk Drives Failure Detection Using A Dynamic Tracking Method

Hard Disk Failure Prediction Based on Blending Ensemble Learning

Intelligent Diagnostic System For Cerebrovascular Diseases Based On A Bayesian Network With Information Gain

A Two-Step Parametric Method for Failure Prediction in Hard Disk Drives.

An Attention-augmented Deep Architecture for Hard Drive Status Monitoring in Large-scale Storage Systems

Proactive Drive Failure Prediction for Cloud Storage System Through Semi-supervised Learning

Multi-Instance Deep Learning Based on Attention Mechanism for Failure Prediction of Unlabeled Hard Disk Drives