Abstract:Predicting hard disk failure effectively and efficiently can prevent the high costs of data loss for data storage systems. Disk failure prediction based on machine learning and artificial intelligence has gained notable attention, because of its good capabilities. Improving the accuracy and performance of disk failure prediction, however, is still a challenging problem. When disk failure is about to occur, the time is limited for the prediction process, including building models and predicting. Faster training would promote the efficiency of model updates, and late predictions not only have no value but also waste resources. To improve both the prediction quality and modeling timeliness, a two-layer classification-based feature selection scheme is proposed in this paper. An attribute filter calculating the importance of attributes was designed, to remove attributes insensitive to failure identification, where importance is gained based on the idea of classification tree models. Furthermore, by determining the correlation between features based on the correlation coefficient, an attribute classification method is proposed. In experiments, the models of machine learning and artificial intelligence were applied, and they included naïve Bayesian, random forest, support vector machine, gradient boosted decision tree, convolutional neural networks, and long short-term memory. The results showed that the proposed technique could improve the prediction accuracy of ML/AI-based hard disk failure prediction models. Specifically, utilizing random forest and long short-term memory with the proposed technique showed the best accuracy. Meanwhile, the proposed scheme could reduce training and prediction latency by 75% and 83%, respectively, in the best case compared with the baseline methods.

Multiple-Instance Learning for Hard Disk Drive Failure Prediction

A Two-Step Parametric Method for Failure Prediction in Hard Disk Drives.

A Failure Prediction Approach Based on BiLSTM and Deep Feature Extractor for Hard Disk Drives.

Multi-Instance Deep Learning Based on Attention Mechanism for Failure Prediction of Unlabeled Hard Disk Drives

Hard Disk Drives Failure Detection Using A Dynamic Tracking Method

Failure Prediction of Hard Disk Drives Based on Adaptive Rao–Blackwellized Particle Filter Error Tracking Method

Failure Prediction of Hard Disk Drives Based on Adaptive Raox2013;Blackwellized Particle Filter Error Tracking Method

Classification Based Hard Disk Drive Failure Prediction: Methodologies, Performance Evaluation and Comparison.

Hard Drive Failure Prediction Using Big Data

Optimizing Efficiency of Machine Learning Based Hard Disk Failure Prediction by Two-Layer Classification-Based Feature Selection

Hard Disk Failure Prediction Based on Blending Ensemble Learning

Hard drive failure prediction using Decision Trees

Hard Drive Failure Prediction Using Classification and Regression Trees

Online Anomaly Detection for Hard Disk Drives Based on Mahalanobis Distance.

Remaining Useful Life Estimation of Hard Disk Drives using Bidirectional LSTM Networks

Proactive drive failure prediction for large scale storage systems

Predicting Hard Drive Failures for Cloud Storage Systems

Hard Disk Failure Prediction Based on Lightgbm with CID.

Proactive Drive Failure Prediction for Cloud Storage System Through Semi-supervised Learning

A Disk Failure Prediction Model for Multiple Issues

Using machine learning to forecast hard drive failures