Abstract:Scalable subsequence matching is critical for supporting analytics on big time series from mining, prediction to hypothesis testing. However, state-of-the-art subsequence matching techniques do not scale well to TB-scale datasets. Not only does index construction become prohibitively expensive, but also the query response time deteriorates quickly as the length of the query subsequence exceeds several 100s of data points. Although Locality Sensitive Hashing (LSH) has emerged as a promising solution for indexing long time series, it relies on expensive hash functions that perform multiple passes over the data and thus is impractical for big time series. In this work, we propose a lightweight distributed indexing framework, called ChainLink, that supports approximate kNN queries over TB-scale time series data. As a foundation of ChainLink, we design a novel hashing technique, called Single Pass Signature (SPS), that successfully tackles the above problem. In particular, we prove theoretically and demonstrate experimentally that the similarity proximity of the indexed subsequences is preserved by our proposed single-pass SPS scheme. Leveraging this SPS innovation, Chainlink then adopts a three-step approach for scalable index building: (1) in-place data re-organization within each partition to enable efficient record-level random access to all subsequences, (2) parallel building of hash-based local indices on top of the re-organized data using our SPS scheme for efficient search within each partition, and (3) efficient aggregation of the local indices to construct a centralized yet highly compact global index for effective pruning of irrelevant partitions during query processing. ChainLink achieves the above three steps in one single map-reduce process. Our experimental evaluation shows that ChainLink indices are compact at less than 2% of dataset size while state-of-the-art index sizes tend to be almost the same size as the dataset. Better still, ChainLink is up to 2 orders of magnitude faster in its index construction time compared to state-of-the-art techniques, while improving both the final query response time by up to 10 fold and the result accuracy by 15%.

KV-match: A Subsequence Matching Approach Supporting Normalization and Time Warping

KV-match: An Efficient Subsequence Matching Approach for Large Scale Time Series.

Mvs-Match: An Efficient Subsequence Matching Approach Based On The Series Synopsis

Multilevel Dynamic Time Warping: A Parameter-Light Method for Fast Time Series Classification

L-Match: A Lightweight and Effective Subsequence Matching Approach.

CIVET: Exploring Compact Index for Variable-Length Subsequence Matching on Time Series

Integrating Data-Driven Segmentation, Local Feature Extraction and Fisher Kernel Encoding to Improve Time Series Classification

A Distributed Multi-level Composite Index for KNN Processing on Long Time Series

The Inherent Time Complexity and an Efficient Algorithm for Subsequence Matching Problem.

ChainLink: Indexing Big Time Series Data For Long Subsequence Matching

Multi-Querying: A Subsequence Matching Approach to Support Multiple Queries.

Exact Indexing for Massive Time Series Databases under Time Warping Distance

Sequence Pattern Matching over Time-Series Data with Temporal Uncertainty

Kernel Sparse Representation for Time Series Classification

TC-Match: Fast Time-Constrained Continuous Subgraph Matching

UniKV: Toward High-Performance and Scalable KV Storage in Mixed Workloads via Unified Indexing

A Data-Adaptive and Dynamic Segmentation Index for Whole Matching on Time Series

Time series indexing by dynamic covering with cross-range constraints

Improving Indexing Approach on Time Series Based on Minimum Bounding Rectangle

Addressing Big Data Time Series: Mining Trillions of Time Series Subsequences Under Dynamic Time Warping

An In-Depth Study of Continuous Subgraph Matching (Complete Version)