Abstract:With the rapid development of cloud computing and big data technologies, storage systems have become a fundamental building block of datacenters, incorporating hardware innovations such as flash solid state drives and non-volatile memories, as well as software infrastructures such as RAID and distributed file systems. Despite the growing popularity and interests in storage, designing and implementing reliable storage systems remains challenging, due to their performance instability and prevailing hardware failures. Proactive prediction greatly strengthens the reliability of storage systems. There are two dimensions of prediction: performance and failure. Ideally, through detecting in advance the slow IO requests, and predicting device failures before they really happen, we can build storage systems with especially low tail latency and high availability. While its importance is well recognized, such proactive prediction in storage systems, on the other hand, is particularly difficult. To move towards predictability of storage systems, various mechanisms and field studies have been proposed in the past few years. In this report, we present a survey of these mechanisms and field studies, focusing on machine learning based black-box approaches. Based on three representative research works, we discuss where and how machine learning should be applied in this field. The strengths and limitations of each research work are also evaluated in detail.

DRAM Failure Prediction in AIOps: EmpiricalEvaluation, Challenges and Opportunities

DRAM Failure Prediction in AIOps: Empirical Evaluation, Challenges and Opportunities

On Workload-Aware DRAM Failure Prediction in Large-Scale Data Centers.

EXPERT: EXPloiting DRAM ERror Types to Improve the Effective Forecasting Coverage in the Field.

A Failure Prediction Approach Based on BiLSTM and Deep Feature Extractor for Hard Disk Drives.

Highly Efficient Memory Failure Prediction using Mcelog-based Data Mining and Machine Learning

Robust Data Preprocessing for Machine-Learning-Based Disk Failure Prediction in Cloud Production Environments

Predicting Node Failures in an Ultra-Large-Scale Cloud Computing Platform

Predicting DRAM-Caused Node Unavailability in Hyper-Scale Clouds

An Introduction to PAKDD CUP 2020 Dataset

Investigating Memory Failure Prediction Across CPU Architectures

System-level Hardware Failure Prediction Using Deep Learning

Multi-Instance Deep Learning Based on Attention Mechanism for Failure Prediction of Unlabeled Hard Disk Drives

Hard Drive Failure Prediction Using Big Data

Prediction of GPU Failures Under Deep Learning Workloads

ZTE-Predictor: Disk Failure Prediction System Based on LSTM

General Feature Selection for Failure Prediction in Large-scale SSD Deployment

Towards Learned Predictability of Storage Systems

Exploit Both SMART Attributes and NAND Flash Wear Characteristics to Effectively Forecast SSD-based Storage Failures in Clusters.

Multiple-Instance Learning for Hard Disk Drive Failure Prediction

Disk failure prediction based on multi-layer domain adaptive learning