Abstract:In the era of big data, log-structured merge-tree (LSM-tree) based key–value stores (KV stores) have become an effective alternative to the traditional relational database owing to their high performance and scalability. KV stores are widely used to manage big unstructured data for data-intensive applications. A compaction operation is triggered to sort data and compact space, but this can degrade write performance, causing write amplification to increase. The compaction process results in extensive data movement and computation. Recently, near-data processing (NDP) models, which place computation close to data sources, aiming to reduce the cost of data movement between the device and the host, have been widely applied to optimize the performance of KV stores. The existing researches on NDP-based KV stores mainly focus on a single NDP-enabled storage device (NDP device for short). In this paper, we present a storage computing architecture with multiple NDP devices named MStore for accelerating compaction performance in LSM-tree based KV stores. MStore can achieve high performance owing to its storage and computing convergence. It employs a multi-column LSM-tree data layout based on key ranges. Compaction tasks on multiple NDP devices are performed in parallel, improving compaction performance. MStore increases the width of the LSM-tree level to reduce the depth, which reduces the execution time and write amplification of compaction. Following the data organization, we design load balancing mechanisms, including storage load balancing by key range adjustment and computation load balancing across compaction tasks. In addition, we develop a batch read interface and a greedy scheduling scheme for user I/Os. MStore aims to fully utilize the computing and I/O resources to boost the compaction performance in KV stores. Compared with single-device NDP systems (such as TStore), MStore with four NDP devices achieves 3.88 × speedup under random-write DB_Bench. When running YCSB-C with a random-write workload, MStore is 4.32 × faster than TStore. The performance of read and scan on MStore is improved by 2.23× and 2.18×, respectively, compared with TStore. These results have strong implications for the implementations of MStore, and their use as replacements for single NDP based KV stores.

Transactional Multi-row Access Guarantee in the Key-Value Store

Supporting Database Applications As a Service

Providing Snapshot+ and Read Committed+ Isolation for Transactional MultiKey Access in the Cloud

Using Paxos to Build a Lightweight, Highly Available Key-Value Data Store

G-Tran: A High Performance Distributed Graph Database with a Decentralized Architecture

UStore: A Distributed Storage with Rich Semantics.

Towards Elastic Transactional Cloud Storage with Range Query Support

Warp: Lightweight Multi-Key Transactions for Key-Value Stores

In Search of a Key Value Store with High Performance and High Availability

gStore-C: A Transactional RDF Store with Light-Weight Optimistic Lock

The Design and Implementation of UniKV for Mixed Key-Value Storage Workloads

UniKV: Toward High-Performance and Scalable KV Storage in Mixed Workloads via Unified Indexing

Studies on LSM-tree Based NoSQL Key-Value Stores

Towards a Non-2PC Transaction Management in Distributed Database Systems

Ustore: an Optimized Storage System for Enterprise Data Warehouses at Unionpay

LibreKV: A Persistent In-Memory Key-Value Store

UniStore: A fault-tolerant marriage of causal and strong consistency (extended version)

An Efficient Design and Implementation of LSM-tree Based Key-Value Store on Open-Channel SSD.

A storage computing architecture with multiple NDP devices for accelerating compaction performance in LSM-tree based KV stores

A Low Disk-Bound Transaction Logging System for In-memory Distributed Data Stores

VeriTxn: Verifiable Transactions for Cloud-Native Databases with Storage Disaggregation.