Abstract:Non- Volatile Memory (NVM) offers low-latency, non-volatility, and byte-addressability, positioning it as a highly promising device for database performance enhancement. Cur-rent research primarily focuses on utilizing LSM-Tree in con-junction with NVM to reduce write amplification and alleviate write stall issues. However, the comprehensive potential of NVM in simultaneously augmenting both read and write performances remains underexplored. And the previous NVM-enhanced LSM-Tree also ignores the sensitivity of NVM to small-grained random reads and writes, which we believe is the key to further improving read and write performance. To address these issues, we propose BushStore, an innovative LSM-Tree variant specifically optimized for NVM. BushStore is designed with a three-level architecture, where the higher levels of BushStore contain a group of immutable, non-clustered B+Trees, replacing traditional SSTables. By storing the non-leaf nodes in the DRAM and the leaf nodes in NVM, and separating the data pages from the indexes, these B+Trees are able to exhibit high performance for diverse read and write operations. Our approach encompasses four key techniques to significantly boost system efficiency: First, we develop novel data structures that localize read/write operations to confined NVM areas, enhancing access speed. Second, we optimize the key-value data handling during flushing and compaction phases, leveraging the superior scanning and sequantial writing capabilities of B+Trees to ex-pedite write and compaction processes. Third, we dynamically adjust the B+Tree sizes, enabling a balanced and optimized flushing and compaction process, thereby improving overall write performance. Fourth, we implement a lazy-delete Cuckoo filtering and lazy-persistent allocation strategy to accelerate query and compaction processes. Evaluations show that BushStore exhibits high performance and scalability under synthetic and real work-loads, and achieves an average performance improvement of 3.3x in random write throughput and 4.3x in random read throughput compared to the state-of-the-art MioDB system.

A Novel Approach For Efficient Accessing Of Small Files In Hdfs: Tlb-Mapfile

A digital library architecture supporting massive small files and efficient replica maintenance.

An archive‐based method for efficiently handling small file problems in HDFS

A Novel Scalable Architecture of Cloud Storage System for Small Files Based on P2P

Storage-Optimization Method for Massive Small Files of Agricultural Resources Based on Hadoop

Small files access efficiency in hadoop distributed file system a case study performed on British library text files

A Proposed Approach for Improving Hadoop Performance for Handling Small Files

A Novel Approach to Improving the Efficiency of Storing and Accessing Small Files on Hadoop: A Case Study by PowerPoint Files

Small Files Problem Resolution via Hierarchical Clustering Algorithm

Data Management Techniques in Hadoop Framework for Handling Small Files: A Survey

BushStore: Efficient B+Tree Group Indexing for LSM-Tree in Non-Volatile Memory

A Novel Approach for Improving Security and Storage Efficiency on HDFS

Impact of Small Files on Hadoop Performance: Literature Survey and Open Points

Small File Read Performance Optimization Based on Redis Cache in Distributed Storage System

Addressing the Small Files Issue in Hadoop

An Approach of Fast Data Manipulation in HDFS with Supplementary Mechanisms

CSFC: A New Centroid Based Clustering Method to Improve the Efficiency of Storing and Accessing Small Files in Hadoop

A Packaging Approach for Massive Amounts of Small Geospatial Files with HDFS.

An Efficient and Compact Indexing Scheme for Large-Scale Data Store.

ALow Cost Algorithm for FastIntelligent Content Delivery Based on Hadoop

TS-Hadoop: Handling Access Skew in MapReduce by Using Tiered Storage Infrastructure