Abstract:Log-structured merge tree (LSM-tree) based key-value stores are widely employed in large-scale storage systems. In compaction, high-level sorted string table files (i.e., SSTables) are merged with low-level overlapping key ranges and sorted for data queries. However, the compaction process incurs write amplification, which degrades system performance particularly under update-intensive workloads. Current optimizations mostly focus on reducing the overload of compaction in the host but rarely exploit parallelisms between the host and the device to fully utilize computing resources in the entire system for optimizing compaction performance. In this study, we propose Co-KV, a Collaborative Key-Value store, to improve compaction performance in LSM-tree-based key-value stores. Co-KV is based on a near-data processing (i.e., NDP) model-enabled storage device. Co-KV exhibits the following advantages: (1) it reduces write amplification and host-side CPU costs using a compaction offloading scheduling between a host computer and an NDP-enabled storage device; (2) it relieves the overload associated with data transfer between the host and the storage device; and (3) it improves the compaction of LSM-tree based key-value stores under update-intensive workloads. We employed an Open Ethernet Driver (OED), which is a real-world NDP platform as the testbed for our experiments. Extensive db_bench evaluations demonstrate that Co-KV achieves overall throughput improvements of approximately 1.75x, CPU cost reductions of approximately 68.1%, and write amplification reductions by up to 50.0% over the state-of-the-art LevelDB. Under YCSB workloads, Co-IN increases throughput by 1.7x similar to 1.9x while decreasing the write amplification and average latency by up to 50.0% and 46.3%, respectively. (C) 2019 Elsevier Inc. All rights reserved.

Hashkv: Enabling Efficient Updates In Kv Storage Via Hashing

Enabling Efficient Updates in KV Storage via Hashing: Design and Performance Evaluation

The Design and Implementation of UniKV for Mixed Key-Value Storage Workloads

KVLB: an In-network Key-Value Load Balancer Using Multi-Valued Hash

UniKV: Toward High-Performance and Scalable KV Storage in Mixed Workloads via Unified Indexing

Differentiated Key-Value Storage Management for Balanced I/O Performance

Co-KV: A Collaborative Key-Value Store Using Near-Data Processing to Improve Compaction for the LSM-tree.

InPlaceKV: In-Place Update Scheme for SSD-based KV Storage Systems under Update-Intensive Worklaods

MoonKV: Optimizing Update-intensive Workloads for NVM-based Key-value Stores

Balancing Garbage Collection vs I/O Amplification using hybrid Key-Value Placement in LSM-based Key-Value Stores

MyWAL: Performance Optimization by Removing Redundant Input/output Stack in Key-Value Store

LearnedKV: Integrating LSM and Learned Index for Superior Performance on SSD

Collaborative Compaction Optimization System Using Near-Data Processing for LSM-tree-based Key-Value Stores

SuccinctKV: a CPU-efficient LSM-tree Based KV Store with Scan-based Compaction

SpacKV: A Pmem-Aware Key-Value Separation Store Based on LSM-Tree

FlashKV

Eukv: Enabling Efficient Updates for Hybrid PM-DRAM Key-Value Store

ChainKV: A Semantics-Aware Key-Value Store for Ethereum System

RadixKV - A Memory Efficient and High Performance Key-Value Store.

Ucleaner: an Efficient Adaptive Garbage Collection Mechanism for KV-Separated LSM-Stores

Workload-Aware Log-Structured Merge Key-Value Store for NVM-SSD Hybrid Storage