Abstract:There is significant interest in examining large datasets using complex domain-specific queries. In many cases, these queries can be accelerated using specialized indexes. Unfortunately, the development of a practical index is difficult, because databases generally require additional features such as updates, concurrency support, crash recovery, etc. There are three major lines of work to alleviate the pain: (1) automatic index composition/tuning which composes indexes out of core data structure primitives to optimize for specific workloads; (2) generalized index templates which generalize common data structures such as B+-trees for custom queries over custom data types, and (3) data structure dynamization frameworks such as the Bentley-Saxe method which converts a static data structure into an updatable data structure with bounded additional query cost. The first two are limited to very specific queries and/or data structures and, thus, are not suitable for building a general index dynamization framework. The last one is more promising in its generality but also has limitations on query types, deletion support, and performance tuning. In this paper, we discuss the limitations of the classic index dynamization techniques and propose a path towards a more general and systematic solution. We demonstrate the viability of our framework by realizing it as a C++20 metaprogramming library and conducting case studies on four example queries with their corresponding static index structures. With this framework, many theoretical/early-stage index designs can easily be extended with support for updates, along with a wide tuning space for query/update performance trade-offs. This allows index designers to focus on efficient data layouts and query algorithms, thereby dramatically narrowing the gap between novel index designs and deployment.

Dynamic Interval Index Structure in Constraint Database Systems

Direct optimization of uncertain structures based on degree of interval constraint violation

AN OPTIMIZED INDEX ALGORITHM BASED ON T-TREE

MC-Tree: Dynamic Index Structure for Partially Clustered Multi-Dimensional Database

Indexing and Querying A Large Database of Typed Intervals.

Effective indexing for dynamic structural graph clustering

Bridging the Gap Between Theory and Practice on Insertion-Intensive Database

DTtree: A Novel Read/Write-Optimized Learned Index for Database Systems.

Dynamising Interval Scheduling: The Monotonic Case.

B-S-Tree: A Self-Tuning Index of Moving Objects

D2-Index: A Dynamic Index Method for Querying XML and Semi-Structured Data

Dynamic Algorithms for Interval Scheduling on a Single Machine.

BITS-Tree-An Efficient Data Structure for Segment Storage and Query Processing

A Concise Concurrent B + -Tree for Persistent Memory

Dynamic algorithms for monotonic interval scheduling problem.

An Effective Directory Index Framework Taking Advantages of Hash Table and B+-Tree

Incremental Mining Large Itemsets with Constraints in Dynamic Databases

Towards Systematic Index Dynamization

Research on Dynamic Indexing Structure for Multi-Dimensional Vectors

The C-ND Tree: a Multidimensional Index for Hybrid Continuous and Non-Ordered Discrete Data Spaces

DC-Tree: Density-Based Clustering Index for Objects in Skewed Distribution