Abstract:Similarity search finds objects that are similar to a given query object based on a similarity metric. As the amount and variety of data continue to grow, similarity search in metric spaces has gained significant attention. Metric spaces can accommodate any type of data and support flexible distance metrics, making similarity search in metric spaces beneficial for many real-world applications, such as multimedia retrieval, personalized recommendation, trajectory analytics, data mining, decision planning, and distributed servers. However, existing studies mostly focus on indexing metric spaces on a single machine, which faces efficiency and scalability limitations with increasing data volume and query amount. Recent advancements in similarity search turn towards distributed methods, while they face challenges including inefficient local data management, unbalanced workload, and low concurrent search efficiency. To this end, we propose DIMS, an efficient Distributed Index for similarity search in Metric Spaces. First, we design a novel three-stage heterogeneous partition to achieve workload balance. Then, we present an effective three-stage indexing structure to efficiently manage objects. We also develop concurrent search methods with filtering and validation techniques that support efficient distributed similarity search. Additionally, we devise a cost-based optimization model to balance communication and computation cost. Extensive experiments demonstrate that DIMS significantly outperforms existing distributed similarity search approaches.

Idisque: Tuning High-Dimensional Similarity Queries in DHT Networks

Improved Dominating Set Indices for Mobile Peer-to-peer Networks

Research on Similarity Search for High-dimensional Data Objects Based on DHT

Dynamic dominant index set for mobile peer-to-peer networks

Distributed Discrete Hashing by Equivalent Continuous Formulation.

Dynamic Indices for Mobile Peer-to-Peer Networks

Secure P2P Topology Based on a Multidimensional DHT Space Mapping

DelQue: A Socially Aware Delegation Query Scheme in Delay-Tolerant Networks

A Lightweight Multidimensional Index for Complex Queries over DHTs

Load balance of the fast similarity search using locality sensitive hashing in structured P2P networks

m-LIGHT: Indexing Multi-Dimensional Data over DHTs

Multidimensional Similarity In-network Query for Large-Scale Sensor Networks

SiMPSON: Efficient Similarity Search in Metric Spaces over P2P Structured Overlay Networks

Distributed Hash table-based distributed subspace clustering algorithm

Multi-dimensional Complex Query Processing over DHT

DIMS: Distributed Index for Similarity Search in Metric Spaces

LIGHT: A Query-Efficient Yet Low-Maintenance Indexing Scheme over DHTs

DIDS: Double Indices and Double Summarizations for Fast Similarity Search

Supporting Dynamic Quantization for High-Dimensional Data Analytics

A Novel Content Distribution Mechanism in DHT Networks

LDHT: Locality-aware Distributed Hash Tables