Abstract:There is significant interest in examining large datasets using complex domain-specific queries. In many cases, these queries can be accelerated using specialized indexes. Unfortunately, the development of a practical index is difficult, because databases generally require additional features such as updates, concurrency support, crash recovery, etc. There are three major lines of work to alleviate the pain: (1) automatic index composition/tuning which composes indexes out of core data structure primitives to optimize for specific workloads; (2) generalized index templates which generalize common data structures such as B+-trees for custom queries over custom data types, and (3) data structure dynamization frameworks such as the Bentley-Saxe method which converts a static data structure into an updatable data structure with bounded additional query cost. The first two are limited to very specific queries and/or data structures and, thus, are not suitable for building a general index dynamization framework. The last one is more promising in its generality but also has limitations on query types, deletion support, and performance tuning. In this paper, we discuss the limitations of the classic index dynamization techniques and propose a path towards a more general and systematic solution. We demonstrate the viability of our framework by realizing it as a C++20 metaprogramming library and conducting case studies on four example queries with their corresponding static index structures. With this framework, many theoretical/early-stage index designs can easily be extended with support for updates, along with a wide tuning space for query/update performance trade-offs. This allows index designers to focus on efficient data layouts and query algorithms, thereby dramatically narrowing the gap between novel index designs and deployment.

Dynamic index selection in data warehouses

Automatic Selection of Bitmap Join Indexes in Data Warehouses

Index and Materialized View Selection in Data Warehouses

Vers l'auto-administration des entrepôts de données

Data Warehouse Native Feature Based OLAP Querying with Keywords

Frequent itemsets mining for database auto-administration

Data Mining-based Materialized View and Index Selection in Data Warehouses

A Join Index for XML Data Warehouses

Robust Index Selection for Stochastic Dynamic Workloads

Sélection simultanée d'index et de vues matérialisées

Query Performance Optimization in XML Data Warehouses

AutoIndex: an Incremental Index Management System for Dynamic Workloads

Les index pour les entrepôts de données : comparaison entre index arbre-B et Bitmap

Materialized View Selection by Query Clustering in XML Data Warehouses

Partial Adaptive Indexing for Approximate Query Answering

Towards Systematic Index Dynamization

An Efficient and Compact Indexing Scheme for Large-Scale Data Store.

"What makes my queries slow?": Subgroup Discovery for SQL Workload Analysis

Innovative Approaches for efficiently Warehousing Complex Data from the Web

Efficient Join Synopsis Maintenance for Data Warehouse.

DWEB: A Data Warehouse Engineering Benchmark