Adaptive Hybrid Indexes

Christoph Anneser,Andreas Kipf,Huanchen Zhang,Thomas Neumann,Alfons Kemper

DOI: https://doi.org/10.1145/3514221.3526121

2022-01-01

Abstract:While index structures are crucial components in high-performance query processing systems, they occupy a large fraction of the available memory. Recently-proposed compact indexes reduce this space overhead and thus speed up queries by allowing the database to keep larger working sets in memory. These compact indexes, however, are slower than performance-optimized in-memory indexes because they adopt encodings that trade performance for memory efficiency. Applying different encodings within a single index might allow optimizing both dimensions at the same time - however, it is not clear which encodings should be applied to which index parts at build time. To take advantage of multiple encodings in one index structure, we present anew framework forming the basis of workload adaptive hybrid indexes which moves encoding decisions to run time instead. By sampling incoming queries adaptively, it tracks accesses to index parts and keeps fine-grained statistics which are used for space-and performance-optimized encoding migrations. We evaluated our framework using B+-trees and tries, and examine the adaptation process and space/performance trade-off for real-world and synthetic workloads. For skewed workloads, our framework can reduce the space by up to 82% while retaining more than 90% of the original performance.