Abstract:Retrieving similar data has drawn many research efforts in the literature due to its importance in data mining, database and information retrieval. This problem is challenging when the data is incomplete. In previous research, data incompleteness refers to the fact that data values for some dimensions are unknown. However, in many practical applications (e.g., data collection by sensor network under bad environment), not only data values but even data dimension information may also be missing, which will make most similarity query algorithms infeasible. In this work, we propose the novel similarity query problem on dimension incomplete data and adopt a probabilistic framework to model this problem. For this problem, users can give a distance threshold and a probability threshold to specify their retrieval requirements. The distance threshold is used to specify the allowed distance between query and data objects and the probability threshold is used to require that the retrieval results satisfy the distance condition at least with the given probability. Instead of enumerating all possible cases to recover the missed dimensions, we propose an efficient approach to speed up the retrieval process by leveraging the inherent relations between query and dimension incomplete data objects. During the query process, we estimate the lower/upper bounds of the probability that the query is satisfied by a given data object, and utilize these bounds to filter irrelevant data objects efficiently. Furthermore, a probability triangle inequality is proposed to further speed up query processing. According to our experiments on real data sets, the proposed similarity query method is verified to be effective and efficient on dimension incomplete data.

Efficient and Effective Similarity Search over Probabilistic Data Based on Earth Mover's Distance

A Progressive Approach for Computing the Earth Mover's Distance

Efficient EMD-based Similarity Search Via Batch Pruning and Incremental Computation (extended Abstract)

MELODY-JOIN: Efficient Earth Mover's Distance Similarity Joins Using MapReduce

A Novel Earth Mover's Distance Methodology for Image Matching with Gaussian Mixture Models.

Distribution-based Similarity Measures for Multi-Dimensional Point Set Retrieval Applications

Efficient approximation of Earth Mover's Distance Based on Nearest Neighbor Search

A linear approximate algorithm for earth mover's distance with thresholded ground distance

Heads-Join: Efficient Earth Mover's Distance Similarity Joins on Hadoop

Probabilistic Similarity Search in Uncertain Time-Series Database

Data-Dependent LSH for the Earth Mover's Distance

Efficient Computation of a Proximity Matching in Spatial Databases

An Efficient Probabilistic Approach for Graph Similarity Search

Probabilistic Similarity Query on Dimension Incomplete Data

Keyword Search over Web Documents Based on Earth Mover's Distance.

The earth mover's distance as a semantic measure for document similarity.

DIMS: Distributed Index for Similarity Search in Metric Spaces

Counts-of-counts similarity for prediction and search in relational data

Efficient Processing of Similarity Search on Uncertain Set-Valued Data

Distance Metric Learning from Uncertain Side Information with Application to Automated Photo Tagging

Exploring Distributional Discrepancy for Multidimensional Point Set Retrieval