Abstract:As a proactive failure tolerant mechanism in large scale cloud storage systems, drive failure prediction can be used to protect data by early warning before real failures of drives, and therefore improve system dependability and cloud storage service quality. At present, solid state drives (SSDs) are generally widely used in cloud storage systems due to their high performance. SSD failures seriously affect the dependability of the system and the quality of service. Existing proactive failure tolerant mechanisms for storage systems are basically aimed at HDD failure detection and use classification technology (Supervised learning), which relies on enough failure data to establish a classification model. However, the low failure rate of SSDs leads to a serious imbalance in the ratio of positive and negative samples, which brings a big challenge for establishing a proactive failure tolerance mechanism for SSDs storage systems by using classification technology. In this paper, we propose a proactive failure tolerance mechanism for SSDs storage systems based on unsupervised technology. It only uses data of normal SSDs to train the failure prediction model, which means that our method is not limited by the imbalance in SSDs data. At the core of our method is the idea to use VAE-LSTM to learn the pattern of normal SSDs, in which case faulty SSDs can be alerted when their patterns are very different from normal ones. Our method can provide early warning of failures, thereby effectively protecting data and improving the quality of cloud storage service. We also propose a drive failure cause location mechanism, which can help operators analyze the modes of failure by providing guiding suggestions. In order to evaluate the effectiveness of our method, we use cross-validation and online testing methods on SSDs data from a technology company. The results show that the FDR and FAR of our method outperform the baselines by 17.25% and 2.39% on average.

A Proactive Failure Tolerant Mechanism for SSDs Storage Systems based on Unsupervised Learning

A Failure Prediction Approach Based on BiLSTM and Deep Feature Extractor for Hard Disk Drives.

Improving the accuracy, adaptability, and interpretability of SSD failure prediction models

Multi-Instance Adversarial Learning Domain Adaptation Network for Failure Prediction of Unlabeled Solid-State Drives

General Feature Selection for Failure Prediction in Large-scale SSD Deployment

Exploit Both SMART Attributes and NAND Flash Wear Characteristics to Effectively Forecast SSD-based Storage Failures in Clusters.

Multi-Instance Deep Learning Based on Attention Mechanism for Failure Prediction of Unlabeled Hard Disk Drives

Towards Learned Predictability of Storage Systems

Improving Productivity and Efficiency of SSD Manufacturing Self-Test Process by Learning-Based Proactive Defect Prediction

Classification Based Hard Disk Drive Failure Prediction: Methodologies, Performance Evaluation and Comparison.

A Novel SSD Fault Detection Method Using GRU-based Sparse Auto-Encoder for Dimensionality Reduction

Layerwise Perturbation-Based Adversarial Training for Hard Drive Health Degree Prediction

Toward Adaptive Disk Failure Prediction Via Stream Mining

Hard Disk Drives Failure Detection Using A Dynamic Tracking Method

Robust Data Preprocessing for Machine-Learning-Based Disk Failure Prediction in Cloud Production Environments

Health Status Assessment and Failure Prediction for Hard Drives with Recurrent Neural Networks

Temporal-Contextual Attention Network for Solid-State Drive Failure Prediction in Data Centers

Optimizing Small-Sample Disk Fault Detection Based on LSTM-GAN Model

Multiple-Instance Learning for Hard Disk Drive Failure Prediction

A One-Class Anomaly Detection Method for Drives Based on Adversarial Auto-Encoder.

An In-Depth Study Of Correlated Failures In Production Ssd-Based Data Centers