A High Performance Management Schema of Metadata Clustering for Large-Scale Data Storage Systems
Xiao Zhongzheng,Chen Ningjiang,Wei Jun,Zhang Wenbo
DOI: https://doi.org/10.7544/issn1000-1239.2015.20131911
2015-01-01
Journal of Computer Research and Development
Abstract:An efficient ,decentralized metadata management schema plays a vital role in large‐scale distributed storage systems . T he Hash‐based partition schema and tree‐based partition schema pay huge cost for expansion ,and are sensitive to changes in cluster .In response to these problems ,CH‐MMS(consistent Hash based metadata management schema) ,is proposed .Virtual MDS (metadata server) is introduced in CH‐MMS ,and good effect for the cluster's load balance is proved .Combining the standby mechanism with lazy‐update policy ,CH‐MMS achieves fast failover and zero migration when the cluster changes .Due to its distributed metadata structure ,CH‐MMS has fast metadata lookup speed .In order to solve the problem that the Hash structure will cause damage to file system hierarchical semantics ,a simple and flexible mechanism based on regular expression matching has been introduced .The following work is presented in the paper :1)Expound the architecture of CH‐MMS ;2)Introduce the core data structure of layout‐table ,virtual MDS and lazy‐update policy ,and their relevant algorithms ;3 ) Qualitatively analyze scalability and fault tolerance . The prototype system and simulation show that ,CH‐MMS is metadata‐balancing and has fast failover ,flexible expansion and zero migration when cluster changes .CH‐MMS can meet the needs of flexible ,efficient metadata management of large‐scale storage systems with increasing data .