Abstract:Similarity search is a crucial task in multimedia retrieval and data mining. Most existing work has modelled this problem as the nearest neighbor (NN) problem, which considers the distance between the query object and the data objects over a fixed set of features. Such an approach has two drawbacks: 1) it leaves many partial similarities uncovered; 2) the distance is often affected by a few dimensions with high dissimilarity. To overcome these drawbacks, we propose the k-n -match problem in this paper.The k-n -match problem models similarity search as matching between the query object and the data objects in n dimensions, where n is a given integer smaller than dimensionality d and these n dimensions are determined dynamically to make the query object and the data objects returned in the answer set match best. The k-n -match query is expected to be superior to the kNN query in discovering partial similarities, however, it may not be as good in identifying full similarity since a single value of n may only correspond to a particular aspect of an object instead of the entirety. To address this problem, we further introduce the frequent k-n -match problem, which finds a set of objects that appears in the k-n -match answers most frequently for a range of n values. Moreover, we propose search algorithms for both problems. We prove that our proposed algorithm is optimal in terms of the number of individual attributes retrieved, which is especially useful for information retrieval from multiple systems. We can also apply the proposed algorithmic strategy to achieve a disk based algorithm for the (frequent) k-n -match query. By a thorough experimental study using both real and synthetic data sets, we show that: 1) the k-n -match query yields better result than the kNN query in identifying similar objects by partial similarities; 2) our proposed method (for processing the frequent k-n -match query) outperforms existing techniques for similarity search in terms of both effectiveness and efficiency.

Efficient EMD-based Similarity Search Via Batch Pruning and Incremental Computation (extended Abstract)

A Progressive Approach for Computing the Earth Mover's Distance

MELODY-JOIN: Efficient Earth Mover's Distance Similarity Joins Using MapReduce

Efficient approximation of Earth Mover's Distance Based on Nearest Neighbor Search

Heads-Join: Efficient Earth Mover's Distance Similarity Joins on Hadoop

Efficient Parallel Partition-Based Algorithms for Similarity Search and Join with Edit Distance Constraints

A novel document similarity measure based on earth mover's distance

Efficient Multimedia Similarity Measurement Using Similar Elements

A Pivotal Prefix Based Filtering Algorithm for String Similarity Search

A similarity computing algorithm for volumetric data sets

An Efficient Partition Based Method for Exact Set Similarity Joins

DIMS: Distributed Index for Similarity Search in Metric Spaces

Accelerating Time Series Similarity Search under Move-Split-Merge Distance Via Dissimilarity Space Embedding

A Bi-metric Framework for Fast Similarity Search

An Efficient Framework for Exact Set Similarity Search Using Tree Structure Indexes.

Technical Report on A Generic Multi-Dimensional Data Generator for Earth Mover ’ s Distance Similarity Analysis

Can we beat the prefix filtering?: an adaptive framework for similarity join and search.

Similarity Search: A Matching Based Approach.

Early Abandoning PrunedDTW and its application to similarity search

The earth mover's distance as a semantic measure for document similarity.

Re-Structuring and Specific Similarity Computation of Electronic Medical Records