Abstract:Data series indexes are necessary for managing and analyzing the increasing amounts of data series collections that are nowadays available. These indexes support both exact and approximate similarity search, with approximate search providing high-quality results within milliseconds, which makes it very attractive for certain modern applications. Reducing the pre-processing (i.e., index building) time and improving the accuracy of search results are two major challenges. DSTree and the iSAX index family are state-of-the-art solutions for this problem. However, DSTree suffers from long index building times, while iSAX suffers from low search accuracy. In this paper, we identify two problems of the iSAX index family that adversely affect the overall performance. First, we observe the presence of a proximity-compactness trade-off related to the index structure design (i.e., the node fanout degree), significantly limiting the efficiency and accuracy of the resulting index. Second, a skewed data distribution will negatively affect the performance of iSAX. To overcome these problems, we propose Dumpy, an index that employs a novel multi-ary data structure with an adaptive node splitting algorithm and an efficient building workflow. Furthermore, we devise Dumpy-Fuzzy as a variant of Dumpy which further improves search accuracy by proper duplication of series. To fully leverage the potential of modern hardware including multicore CPUs and Solid State Drives (SSDs), we parallelize Dumpy to DumpyOS with sophisticated indexing and pruning-based querying algorithms. An optimized approximate search algorithm, DumpyOS-F that prominently improves the search accuracy without violating the index, is also proposed. Experiments with a variety of large, real datasets demonstrate that the Dumpy solutions achieve considerably better efficiency, scalability and search accuracy than its competitors. DumpyOS further improves on Dumpy, by delivering several times faster index building and querying, and DumpyOS-F improves the search accuracy of Dumpy-Fuzzy without the additional space cost of Dumpy-Fuzzy. This paper is an extension of the previously published SIGMOD paper [81].

AdaIndex: an Adaptive Index Structure for Fast Similarity Search in Metric Spaces

DIMS: Distributed Index for Similarity Search in Metric Spaces

Novel High-Dimensional Indexing Structure Based on Dual-Distance Metric

Indexing Metric Spaces for Exact Similarity Search

DESIRE: An Efficient Dynamic Cluster-based Forest Indexing for Similarity Search in Multi-Metric Spaces

Efficient Metric Indexing for Similarity Search

DIDS: Double Indices and Double Summarizations for Fast Similarity Search

iDistance: An adaptive B+-tree based indexing method for nearest neighbor search

A Multidimensional Index Structure for Fast Similarity Retrieval

A Learned Index for Exact Similarity Search in Metric Spaces

D(k)-index: an Adaptive Structural Summary for Graph-Structured Data

Dynamic Indexing for Multidimensional Non-Ordered Discrete Data Spaces Using a Data-Partitioning Approach

Angle-Tree: A New Index Structure for High-Dimensional Point Data

D(k)-index

Indexing high-dimensional data for efficient in-memory similarity search

DumpyOS: A data-adaptive multi-ary index for scalable data series similarity search

An Adaptive Index Structure for Similarity Search in Large Image Databases

The SuperM-Tree: Indexing metric spaces with sized objects

A High Dimensional Index Based on Relative Distance Hashing Method

SAM: A Spatial-Aware Learned Index for Disk-Based Multi-dimensional Search

Enabling Structural Summaries for Efficient Update and Workload Adaptation