Abstract:In the era of big data, log-structured merge-tree (LSM-tree) based key–value stores (KV stores) have become an effective alternative to the traditional relational database owing to their high performance and scalability. KV stores are widely used to manage big unstructured data for data-intensive applications. A compaction operation is triggered to sort data and compact space, but this can degrade write performance, causing write amplification to increase. The compaction process results in extensive data movement and computation. Recently, near-data processing (NDP) models, which place computation close to data sources, aiming to reduce the cost of data movement between the device and the host, have been widely applied to optimize the performance of KV stores. The existing researches on NDP-based KV stores mainly focus on a single NDP-enabled storage device (NDP device for short). In this paper, we present a storage computing architecture with multiple NDP devices named MStore for accelerating compaction performance in LSM-tree based KV stores. MStore can achieve high performance owing to its storage and computing convergence. It employs a multi-column LSM-tree data layout based on key ranges. Compaction tasks on multiple NDP devices are performed in parallel, improving compaction performance. MStore increases the width of the LSM-tree level to reduce the depth, which reduces the execution time and write amplification of compaction. Following the data organization, we design load balancing mechanisms, including storage load balancing by key range adjustment and computation load balancing across compaction tasks. In addition, we develop a batch read interface and a greedy scheduling scheme for user I/Os. MStore aims to fully utilize the computing and I/O resources to boost the compaction performance in KV stores. Compared with single-device NDP systems (such as TStore), MStore with four NDP devices achieves 3.88 × speedup under random-write DB_Bench. When running YCSB-C with a random-write workload, MStore is 4.32 × faster than TStore. The performance of read and scan on MStore is improved by 2.23× and 2.18×, respectively, compared with TStore. These results have strong implications for the implementations of MStore, and their use as replacements for single NDP based KV stores.

Adaptive Lower-Level Driven Compaction to Optimize LSM-Tree Key-Value Stores

FPGA-Accelerated Compactions for LSM-based Key-Value Store.

Constructing and Analyzing the LSM Compaction Design Space (Updated Version)

BushStore: Efficient B+Tree Group Indexing for LSM-Tree in Non-Volatile Memory

<i>SA-LSM</i>: Optimize Data Layout for LSM-tree Based Storage using Survival Analysis

SA-LSM

Studies on LSM-tree Based NoSQL Key-Value Stores

Transactional Multi-row Access Guarantee in the Key-Value Store

Autumn: A Scalable Read Optimized LSM-tree based Key-Value Stores with Fast Point and Range Read Speed

SplitDB: Closing the Performance Gap for LSM-Tree-Based Key-Value Stores

A storage computing architecture with multiple NDP devices for accelerating compaction performance in LSM-tree based KV stores

An Efficient Design and Implementation of LSM-tree Based Key-Value Store on Open-Channel SSD.

Accelerating LSM-Tree with the Dentry Management of File System

LUDA: Boost LSM Key Value Store Compactions with GPUs

Towards flexibility and robustness of LSM trees

vLSM: Low tail latency and I/O amplification in LSM-based KV stores

Time-tired compaction: An elastic compaction scheme for LSM-tree based time-series database

An Optimization of Key-Value Store Based on Segmented LSM-Tree

On Reducing Space Amplification with Multi-Column Compaction in Apache IoTDB

LSM-based Storage Techniques: A Survey