Abstract:As a proactive failure tolerant mechanism in large scale cloud storage systems, drive failure prediction can be used to protect data by early warning before real failures of drives, and therefore improve system dependability and cloud storage service quality. At present, solid state drives (SSDs) are generally widely used in cloud storage systems due to their high performance. SSD failures seriously affect the dependability of the system and the quality of service. Existing proactive failure tolerant mechanisms for storage systems are basically aimed at HDD failure detection and use classification technology (Supervised learning), which relies on enough failure data to establish a classification model. However, the low failure rate of SSDs leads to a serious imbalance in the ratio of positive and negative samples, which brings a big challenge for establishing a proactive failure tolerance mechanism for SSDs storage systems by using classification technology. In this paper, we propose a proactive failure tolerance mechanism for SSDs storage systems based on unsupervised technology. It only uses data of normal SSDs to train the failure prediction model, which means that our method is not limited by the imbalance in SSDs data. At the core of our method is the idea to use VAE-LSTM to learn the pattern of normal SSDs, in which case faulty SSDs can be alerted when their patterns are very different from normal ones. Our method can provide early warning of failures, thereby effectively protecting data and improving the quality of cloud storage service. We also propose a drive failure cause location mechanism, which can help operators analyze the modes of failure by providing guiding suggestions. In order to evaluate the effectiveness of our method, we use cross-validation and online testing methods on SSDs data from a technology company. The results show that the FDR and FAR of our method outperform the baselines by 17.25% and 2.39% on average.

An In-Depth Study Of Correlated Failures In Production Ssd-Based Data Centers

SSD Failures in Datacenters

Measurement and Analysis of SSD Reliability Data Based on Accelerated Endurance Test

Boosting Correlated Failure Repair in SSD Data Centers

The Life and Death of SSDs and HDDs: Similarities, Differences, and Prediction Models

Exploit Both SMART Attributes and NAND Flash Wear Characteristics to Effectively Forecast SSD-based Storage Failures in Clusters.

Improving the accuracy, adaptability, and interpretability of SSD failure prediction models

General Feature Selection for Failure Prediction in Large-scale SSD Deployment

Investigating Power Outage Effects on Reliability of Solid-State Drives

An Experimental Study of Redundant Array of Independent SSDs and Filesystems

Review on HDD-SSD hybrid storage

A Modeling Framework for Reliability of Erasure Codes in SSD Arrays

Predicting solid state drive reliability

On Correlated Failures in Survivable Storage Systems

How does SSD Cluster Perform for Distributed File Systems: An Empirical Study

Lifespan Analysis for Redundant Array of Independent Module Based Solid State Drives

Chameleon: an Adaptive Wear Balancer for Flash Clusters

NVMe SSD Failures in the Field: the Fail-Stop and the Fail-Slow

Modeling SSD RAID reliability under general settings

A Proactive Failure Tolerant Mechanism for SSDs Storage Systems based on Unsupervised Learning

Evaluating Reliability of SSD-Based I/O Caches in Enterprise Storage Systems