Abstract:The massive amounts of time series data continuously generated and collected by applications warrant the need for large scale distributed time series processing systems. Indexing plays a critical role in speeding up time series similarity queries on which various analytics and applications rely. However, the state-of-the-art indexing techniques, which are iSAX-based structures, do not scale well due to the small adopted fan-out (binary) that leads to a highly deep index tree, and the expensive search cost through many internal nodes. More seriously, the iSAX character-level cardinality adopted by these indices suffers from a poor maintenance of the proximity relationships among the time series objects, which leads to severe accuracy degradation for approximate similarity queries. In this paper, we propose the TARDIS distributed indexing framework to overcome the aforementioned limitations. TARDIS introduces a novel iSAX index tree that is based on a new word-level variable cardinality. The proposed index ensures compact structure, efficient search and comparison, and good preservation of the similarity relationships. TARDIS is suitable for indexing and querying billion-scale time series datasets. TARDIS is composed of one centralized global index and local distributed indices-one per each data partition across the cluster. TARDIS uses both the global and local indices to efficiently support exact match and kNN approximate queries. The system is implemented using Apache Spark, and extensive experiments are conducted on benchmark and real-world datasets. Evaluation results demonstrate that for over one billion time series dataset (TB scale), the construction of a clustered index is about 83% faster than the existing techniques. Moreover, the average response time of exact match queries is decreased by 50%, and the accuracy of the kNN approximate queries has increased more than 10 fold (from 3% to 40%) compared to the existing techniques.

Distributed indices for accessing distributed data

Dynamic Indices for Mobile Peer-to-Peer Networks

Implementation of large-scale distributed information retrieval system

An Error-Resilient and Tunable Distributed Indexing Scheme for Wireless Data Broadcast

Adaptive Indexing for Distributed Array Processing

Efficient B-tree Based Indexing for Cloud Data Processing.

A practical scalable distributed B-tree

An Efficient and Compact Indexing Scheme for Large-Scale Data Store.

A Framework for Supporting DBMS-like Indexes in the Cloud

New Distributed Spatial Query Optimization Approach by Using Query Analyzer

Distributed Shortest Distance Labeling on Large-Scale Graphs

Fully Distributed R-Tree For Efficient Range Query Dissemination In Peer-To-Peer Spatial Data Grid

Indexing multi-dimensional data in a cloud system.

A scalable distributed information management system

Designing Distributed Tree-based Index Structures for Fast RDMA-capable Networks

TARDIS: Distributed Indexing Framework for Big Time Series Data

Design of structured data storage index algorithm based on P2P

Data Independent Method of Constructing Distributed LSH for Large-Scale Dynamic High-Dimensional Indexing

Effective Data Distribution And Reallocation Strategies For Fast Query Response In Distributed Query-Intensive Data Environments

Dumpy: A Compact and Adaptive Index for Large Data Series Collections

A survey of P2P multidimensional indexing structures