Abstract:Most time series data mining algorithms use similarity search as a core subroutine, and thus the time taken for similarity search is the bottleneck for virtually all time series data mining algorithms, including classification, clustering, motif discovery, anomaly detection, and so on. The difficulty of scaling a search to large datasets explains to a great extent why most academic work on time series data mining has plateaued at considering a few millions of time series objects, while much of industry and science sits on billions of time series objects waiting to be explored. In this work we show that by using a combination of four novel ideas we can search and mine massive time series for the first time. We demonstrate the following unintuitive fact: in large datasets we can exactly search under Dynamic Time Warping (DTW) much more quickly than the current state-of-the-art Euclidean distance search algorithms. We demonstrate our work on the largest set of time series experiments ever attempted. In particular, the largest dataset we consider is larger than the combined size of all of the time series datasets considered in all data mining papers ever published. We explain how our ideas allow us to solve higher-level time series data mining problems such as motif discovery and clustering at scales that would otherwise be untenable. Moreover, we show how our ideas allow us to efficiently support the uniform scaling distance measure, a measure whose utility seems to be underappreciated, but which we demonstrate here. In addition to mining massive datasets with up to one trillion datapoints, we will show that our ideas also have implications for real-time monitoring of data streams, allowing us to handle much faster arrival rates and/or use cheaper and lower powered devices than are currently possible.

Discovering similar time-series patterns with fuzzy clustering and DTW methods

Dynamic Time Warping under Product Quantization, with Applications to Time-Series Data Similarity Search

Multilevel Dynamic Time Warping: A Parameter-Light Method for Fast Time Series Classification

Research on methodology of time serial data mining

A clustering algorithm for distributed time-series data

Segment-based similar time series search

Data Mining Based on Fluctuation Feature in Time Series

An Algorithm for Mining Association Patterns Between Two Time Series and Application in Finance

Time Series Similar Mining Based on Wavelet Transformation

A NEW ALGORITHM OF SIMILARITY MINING IN TIME SERIES DATA

An Approach to Discovering Multi-Temporal Patterns and Its Application to Financial Databases

Addressing Big Data Time Series: Mining Trillions of Time Series Subsequences Under Dynamic Time Warping

Time series forecasting based on similarity-based clustering algorithm

Features Extraction And Correlation Analysis Of Stock Index

An Algorithm for Similar Sub-patterns Discovery from Time Series

Data Mining in Time Series: Current Study and Future Trend

Speed Up Similarity Search of Time Series under Dynamic Time Warping

A method for measuring similarity of time series based on series decomposition and dynamic time warping

Similarity Discovery Techniques in Temporal Data Mining

An Adaptive Filtering Technique For Time Series Search

An Algorithm of Finding Similar Subpattern Based on Time Series