Abstract:Erasure coding has been widely used in commodity datacenter to tolerate faults, due to its ability to simultaneously provide high storage space utilization and data reliability. However, when data loss occurs, the extra data decoding and traffic overhead makes it difficult to improve their data repair efficiency, limiting their further application in the hot data storage systems. In this paper, we proposed an FPGA-based Data Repair accelerating scheme (FPGA-ECDR) for erasure-coded storage system, which employs the Cauchy Reed-Solomon(CRS) code, to overcome the aforementioned limitation. In FPGA-ECDR, multiple modules are designed to work collaboratively, enhancing the efficiency of data flow and ensuring the reliability of the data repair process. Then, a CRS decoding algorithm based on check matrix is used to reduce the complexity of matrix inversion in the decoding process, and hardware acceleration of the algorithm is realized by FPGA. Moreover, we proposed a Hierarchical Parallel Decoding Structure (HPDS) to optimize cache data reading timing and XOR logic operations in the decoding process. HPDS can effectively reduce the impact of Column Address Strobe (CAS) latency and improve repair efficiency. Finally, we conducted FPGA board-level verification of the proposed scheme, testing on CRS codes with different data sizes and parameters. The experimental results show that compared with the schemes of the current mainstream open-source erasure coding library Jerasure and Intel ISA-L acceleration library, as well as the Xilinx RS code decoding acceleration scheme based on the same FPGA platform, our proposed scheme has lower decoding latency and can improve the data decoding rate by 3.2 to 148.5 times and enhances the repair throughput by up to 21.4 times in fault-tolerant storage system.

NetEC: Accelerating Erasure Coding Reconstruction with In-Network Aggregation

Towards In-network Acceleration of Erasure Coding

Accelerating erasure coding by exploiting multiple repair paths in distributed storage systems

NetFEC: In-network FEC Encoding Acceleration for Latency-sensitive Multimedia Applications

Demand-Aware Erasure Coding for Distributed Storage Systems

FPGA-Accelerated Erasure Coding Encoding in Ceph Based on an Efficient Layered Strategy

An Adaptive Erasure-Coded Storage Scheme with an Efficient Code-Switching Algorithm.

A Solution to the Network Challenges of Data Recovery in Erasure-coded Distributed Storage Systems: A Study on the Facebook Warehouse Cluster

Data repair accelerating scheme for erasure-coded storage system based on FPGA and hierarchical parallel decoding structure

Parallelized In-Network Aggregation for Failure Repair in Erasure-Coded Storage Systems

Fast Recovery Techniques for Erasure-coded Clusters in Non-uniform Traffic Network.

XORing Elephants: Novel Erasure Codes for Big Data

EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation

Joint Source Selection And Transfer Optimization For Erasure Coding Storage System

Repair Tree: Fast Repair for Single Failure in Erasure-Coded Distributed Storage Systems

Rack-Aware Regenerating Codes with Multiple Erasure Tolerance

Elastic Resource Allocation for Coded Distributed Computing Over Heterogeneous Wireless Edge Networks

ESetStore: An Erasure-Coded Storage System With Fast Data Recovery

CXL over Ethernet: A Novel FPGA-based Memory Disaggregation Design in Data Centers

Accelerating XOR-based Erasure Coding using Program Optimization Techniques

Deep Reinforcement Learning-Based Network Routing Technology for Data Recovery in Exa-Scale Cloud Distributed Clustering Systems