Abstract:Exact set similarity join, which finds all the similar set pairs from two collections of sets, is a fundamental problem with a wide range of applications. The existing solutions for set similarity join follow a filtering-verification framework, which generates a list of candidate pairs through scanning indexes in the filtering phase, and reports those similar pairs in the verification phase. Though much research has been conducted on this problem, set relations, which we find out is quite effective on improving the algorithm efficiency through computational cost sharing, have never been studied. Therefore, in this paper, instead of considering each set individually, we explore the set relations in different levels to reduce the overall computational costs. First, it has been shown that most of the computational time is spent on the filtering phase, which can be quadratic to the number of sets in the worst case for the existing solutions. Thus we explore index-level set relations to reduce the filtering cost to be linear to the size of the input while keeping the same filtering power. We achieve this by grouping related sets into blocks in the index and skipping useless index probes in joins. Second, we explore answer-level set relations to further improve the algorithm based on the intuition that if two sets are similar, their answers may have a large overlap. We derive an algorithm which incrementally generates the answer of one set from an already computed answer of another similar set rather than compute the answer from scratch to reduce the computational cost. Finally, we conduct extensive performance studies using 21 real datasets with various data properties from a wide range of domains. The experimental results demonstrate that our algorithm outperforms all the existing algorithms across all datasets and can achieve more than an order of magnitude speedup against the state- of-the-art algorithms.

Computing Probability Threshold Set Similarity On Probabilistic Sets

Similarity Query Processing for Probabilistic Sets

Finding Probabilistic k-Skyline Sets on Uncertain Data.

Efficient Processing of Similarity Search on Uncertain Set-Valued Data

Computing Skyline Probabilities on Uncertain Time Series.

Aggregate Queries on Constrained Probabilistic Similarity Join Pairs

Probabilistic Similarity Query on Dimension Incomplete Data

An Efficient Partition Based Method for Exact Set Similarity Joins

Leveraging Set Relations in Exact and Dynamic Set Similarity Join

Overlap Set Similarity Joins with Theoretical Guarantees.

A Similarity Computing Algorithm for Proteins

Optimizing Quality for Probabilistic Skyline Computation and Probabilistic Similarity Search

Computing Event Probability in Probabilistic Databases

On Similarity Measures Between Pythagorean Fuzzy Sets Derived from Overlap and Grouping Functions

TST: Threshold Based Similarity Transitivity Method in Collaborative Filtering with Cloud Computing

GPH: Similarity Search in Hamming Space.

Efficiently Answering Probabilistic Threshold Top-k Queries on Uncertain Data

Assessing the Similarity of Quantum Images Based on Probability Measurements

Evaluation of Probability Transformations of Belief Functions for Decision Making.

Approaches to Three-Way Decisions Based on the Evaluation of Probabilistic Linguistic Terms Sets

An Efficient Probabilistic Approach for Graph Similarity Search