A Dynamic Replica Strategy Based on Markov Model for Hadoop Distributed File System (HDFS)

Kaiyang Qu,Luoming Meng,Yang
DOI: https://doi.org/10.1109/ccis.2016.7790280
2016-01-01
Abstract:In this paper, we describe DRS, a dynamic replica strategy based on improved Markov model for Hadoop Distributed File System (HDFS). DRS can dynamically increase or reduce the replicas with the data becoming hot or cold. It includes two parts: a dynamic replica adjustment strategy (DRAS) and a homogeneous replica placement strategy (HRPS). First, DRS constructs a transition probability matrix based on the file accessing times in a period and calculates the stationary probability distribution of the system. It utilizes the results to distinguish different data types, and then increases extra replicas for hot data, clean up these extra replicas when the data cool down. Finally, these replicas are homogeneously distributed across the racks and nodes. The experimental results show that DRS is effective, and it is obviously superior to static replica strategy of HDFS.
What problem does this paper attempt to address?