Abstract:Cloud storage systems often face the issues of failure and straggler nodes. Failure is characterized as a fail-stop scenario, which refers to disk failures that can result in significant data unavailability. Straggler nodes are typically those with heavy workloads or poor performance. Usually, both failure and straggler nodes coexist, posing a significant challenge to data availability in storage systems. In such failure scenarios, parallel recovery and straggler recovery methods are commonly used as separate approaches for data recovery. However, parallel recovery methods encounter bottlenecks on the recovery path due to the presence of straggler nodes. Meanwhile, straggler recovery methods face the challenge of lacking available recovery paths in cases of multiple node failures. Scenarios involving both multiple failures and stragglers are common, yet there is a lack of efficient recovery methods for these situations. In this paper, we focus on scenarios involving video data, which occupies a significant portion of cloud storage systems, to address the above issues. We propose a Hybrid Global Graph-based Recovery (HGR) method that integrates parallel and straggler recovery approaches into a single global graph. The key idea of HGR is to construct a global graph that includes global node parameter information, enabling comprehensive coordination. We partition the global graph into two subgraphs: one containing straggler nodes and the other containing failure nodes. Resources are efficiently allocated to each subgraph to schedule recovery tasks in parallel. For data that presents significant recovery challenges, exhibits poor parallelism, has substantial tail latency, or exceeds fault tolerance limits, we employ approximate recovery methods. To demonstrate HGR's effectiveness, we conducted several experiments. The results indicate that HGR can reduce recovery time by up to 45.06% and improve I/O throughput by as much as 1.79x compared to state-of-the-art recovery methods.

GRPU: an Efficient Graph-based Cross-Rack Parallel Update Scheme for Cloud Storage Systems

A Graph-Assisted Out-of-Place Update Scheme for Erasure Coded Storage Systems.

Study on Data Redundancy Scheme in Kademlia Cloud Storage System

Cross-Rack Update Bandwidth for Distributed Storage Systems

XHR- Code: An Efficient Wide Stripe Erasure Code to Reduce Cross- Rack Overhead in Cloud Storage Systems

FAGR: an Efficient File-aware Graph Recovery Scheme for Erasure Coded Cloud Storage Systems.

Greedy Transfer Planning Search for Improving Repair Throughput of RDP-like Coded Storage Clusters

An Erasure Code-based Approach to Improve Data Recovery and Update Capability

Fine-grained scheduling policy based on erasure code

Rack-Scaling: An efficient rack-based redistribution method to accelerate the scaling of cloud disk arrays

A novel video replica placement strategy for storage cloud-based CDN

RCS: A Redirection Computational Scheduler to Accelerate Straggler Recovery for Erasure Coded Cloud Storage System

When Queueing Meets Coding: Optimal-latency Data Retrieving Scheme in Storage Clouds

Efficient Load Balance of the I/O Operations for Large-Scale Erasure-Coded Key-Value Storage Systems in Higher Education

HGR: A Hybrid Global Graph-Based Recovery Approach for Cloud Storage Systems with Failure and Straggler Nodes

Demand-Aware Erasure Coding for Distributed Storage Systems

Study on Redundant Strategies in Peer to Peer Cloud Storage Systems

CaCo: an Efficient Cauchy Coding Approach for Cloud Storage Systems

Efficient Parity Update For Scaling Raid-Like Storage Systems

Tight Lower Bound on Cross-Rack Update Bandwidth and Explicit Constructions

EC-Fusion: an Efficient Hybrid Erasure Coding Framework to Improve Both Application and Recovery Performance in Cloud Storage Systems