HMT: A Hybrid Mitigating and Transferring Approach on I/O Throughput Degradation for Erasure Coded Storage Systems

Piao Hu,Huangzhen Xue,Chentao Wu,Jie Li,Minyi Guo

DOI: https://doi.org/10.1145/3673038.3673052

2024-01-01

Abstract:In cloud storage systems with erasure coding (EC), increased demand for data services and EC-based data recovery lead to high volumes of concurrent I/O requests, potentially causing network congestion or server overload. Network congestion or node overload significantly reduces I/O throughput and data parallelism. Various methods have been proposed to address these issues, such as fine-grained data packet partitioning, I/O scheduling, and transfer reading. However, these methods may not be effective in different scenarios. For instance, even if most I/O paths are relieved, data may still remain inaccessible. To address this problem, we propose a novel hybrid approach called HMT, which combines mitigation and transfer techniques aimed at maintaining high I/O throughput for cloud storage systems. The key idea behind HMT is to pick an appropriate method with low overhead. Mitigating-based methods are used when light node load or congestion causes minor I/O throughput declines. Conversely, transfer-based methods are preferred when heavy load or congestion leads to significant I/O throughput reductions. HMT also provides a transformation strategy between mitigation and transfer-based techniques. To demonstrate the effectiveness of our approach, we conducted several experiments. The experimental results demonstrate that, in scenarios with dynamically changing loads, HMT can reduce the time to read 95% of files and the average response time by up to 49.02% and 50.53%, respectively, compared to state-of-the-art methods. Meanwhile, it also increases throughput by up to 1.70x and optimizes load balancing by up to 36.63%.