Abstract:Similarity search finds objects that are similar to a given query object based on a similarity metric. As the amount and variety of data continue to grow, similarity search in metric spaces has gained significant attention. Metric spaces can accommodate any type of data and support flexible distance metrics, making similarity search in metric spaces beneficial for many real-world applications, such as multimedia retrieval, personalized recommendation, trajectory analytics, data mining, decision planning, and distributed servers. However, existing studies mostly focus on indexing metric spaces on a single machine, which faces efficiency and scalability limitations with increasing data volume and query amount. Recent advancements in similarity search turn towards distributed methods, while they face challenges including inefficient local data management, unbalanced workload, and low concurrent search efficiency. To this end, we propose DIMS, an efficient Distributed Index for similarity search in Metric Spaces. First, we design a novel three-stage heterogeneous partition to achieve workload balance. Then, we present an effective three-stage indexing structure to efficiently manage objects. We also develop concurrent search methods with filtering and validation techniques that support efficient distributed similarity search. Additionally, we devise a cost-based optimization model to balance communication and computation cost. Extensive experiments demonstrate that DIMS significantly outperforms existing distributed similarity search approaches.

Metric Similarity Joins Using Mapreduce (Extended Abstract)

Metric Similarity Joins Using MapReduce

An Efficient MapReduce Algorithm for Similarity Join in Metric Spaces

Scalable Metric Similarity Join Using MapReduce

Join Query Optimization Based on MapReduce under Skewed Data

Improving Distributed Similarity Join in Metric Space with Error-bounded Sampling

Improve Semantic Web Services Discovery Through Similarity Search in Metric Space

MELODY-JOIN: Efficient Earth Mover's Distance Similarity Joins Using MapReduce

Heads-Join: Efficient Earth Mover's Distance Similarity Joins on Hadoop

Massjoin: A Mapreduce-Based Method for Scalable String Similarity Joins

Distributed Similarity Queries in Metric Spaces

SiMPSON: Efficient Similarity Search in Metric Spaces over P2P Structured Overlay Networks

Efficient Similarity Search on Quasi-Metric Graphs

An Efficient Partition Based Method for Exact Set Similarity Joins

What Makes Objects Similar: A Unified Multi-Metric Learning Approach.

Efficient Processing of k Nearest Neighbor Joins using MapReduce

DIMS: Distributed Index for Similarity Search in Metric Spaces

MSQL: Efficient Similarity Search in Metric Spaces Using SQL.

Unified Spatial Intersection Algorithms Based on Conformal Geometric Algebra

Unconventional application of k-means for distributed approximate similarity search