Abstract:Learned Index, which utilizes effective machine learning models to accelerate locating sorted data positions, has gained increasing attention in many big data scenarios. Using efficient learned models, the learned indexes build large nodes and flat structures, thereby greatly improving the performance. However, most of the state-of-the-art learned indexes are designed for DRAM, and there is hence an urgent need to enable high-performance learned indexes for emerging Non-Volatile Memory (NVM). In this paper, we first evaluate and analyze the performance of the existing learned indexes on NVM. We discover that these learned indexes encounter severe write amplification and write performance degradation due to the requirements of maintaining large sorted/semi-sorted data nodes. To tackle the problems, we propose a novel three-tiered architecture of write-optimized persistent learned index, which is named WIPE , by adopting unsorted fine-granularity data nodes to achieve high write performance on NVM. Thereinto, we devise a new root node construction algorithm to accelerate searching numerous small data nodes. The algorithm ensures stable flat structure and high read performance in large-size datasets by introducing an intermediate layer (i.e., index nodes) and achieving accurate prediction of index node positions from the root node. Our extensive experiments on Intel DCPMM show that WIPE can improve write throughput and read throughput by up to 3.9 × and 7 ×, respectively, compared to the state-of-the-art learned indexes. Also, WIPE can recover from a system crash in ∼ 18 ms . WIPE is free as an open-source software package 1 .

Revisiting Learned Index with Byte-addressable Persistent Storage

When Learned Indexes Meet Persistent Memory: the Analysis and the Optimization

PLIN: A Persistent Learned Index for Non-Volatile Memory with High Performance and Instant Recovery.

LIFM: A Persistent Learned Index for Flash Memory

Revisiting Persistent Indexing Structures on Intel Optane DC Persistent Memory

WIPE: a Write-Optimized Learned Index for Persistent Memory

Exploiting Persistent CPU Cache for Scalable Persistent Hash Index

An efficient labeled memory system for learned indexes

The past, present and future of indexing on persistent memory

Revisiting Secondary Indexing in LSM-based Storage Systems with Persistent Memory.

DiffLex: A High-Performance, Memory-Efficient and NUMA-Aware Learned Index Using Differentiated Management.

A Simple Yet High-Performing On-disk Learned Index: Can We Have Our Cake and Eat it Too?

An Efficient Flattened Index Structure with Lazy Restructuring and Hotness Awareness

Making In-Memory Learned Indexes Efficient on Disk

Learned Index: A Comprehensive Experimental Evaluation

Revisiting the Design of LSM-tree Based OLTP Storage Engine with Persistent Memory

A Novel Index-Organized Storage Model for Hybrid DRAM-PM Main Memory Database Systems

Alter: Towards Optimizing Persistent Indexes in Hybrid Memory Systems

A Scalable Learned Index Scheme in Storage Systems

SALI: A Scalable Adaptive Learned Index Framework based on Probability Models

BPTree: an Optimized Index with Batch Persistence on Optane DC PM