Abstract:Image retrieval is crucial in robotics and computer vision, with downstream applications in robot place recognition and vision-based product recommendations. Modern retrieval systems face two key challenges: scalability and efficiency. State-of-the-art image retrieval systems train specific neural networks for each dataset, an approach that lacks scalability. Furthermore, since retrieval speed is directly proportional to embedding size, existing systems that use large embeddings lack efficiency. To tackle scalability, recent works propose using off-the-shelf foundation models. However, these models, though applicable across datasets, fall short in achieving performance comparable to that of dataset-specific models. Our key observation is that, while foundation models capture necessary subtleties for effective retrieval, the underlying distribution of their embedding space can negatively impact cosine similarity searches. We introduce Autoencoders with Strong Variance Constraints (AE-SVC), which, when used for projection, significantly improves the performance of foundation models. We provide an in-depth theoretical analysis of AE-SVC. Addressing efficiency, we introduce Single-shot Similarity Space Distillation ((SS)$_2$D), a novel approach to learn embeddings with adaptive sizes that offers a better trade-off between size and performance. We conducted extensive experiments on four retrieval datasets, including Stanford Online Products (SoP) and Pittsburgh30k, using four different off-the-shelf foundation models, including DinoV2 and CLIP. AE-SVC demonstrates up to a $16\%$ improvement in retrieval performance, while (SS)$_2$D shows a further $10\%$ improvement for smaller embedding sizes.

Distributed Architecture for Large Scale Image-Based Search

Implementation of large-scale distributed information retrieval system

Learning to Distribute Vocabulary Indexing for Scalable Visual Search

Efficient Indexing for Large Scale Visual Search

Allocating images and selecting image collections for distributed visual search

Efficient Indexing for Large-Scale Image Search

A Scalable Distributed Architecture for Intelligent Vision System.

Fine-Grained Image Search

Large Scale Nearest Neighbors Search Based on Neighborhood Graph.

Interactive Browsing of Large Images on Multi-Projector Display Wall System

Multi-Scale Visual Words For Object-Based Web Image Search

Modeling Image Data for Effective Indexing and Retrieval in Large General Image Databases.

Hierarchical Indexing Scheme for Fast Search in Large-Scale Image Database

Large scale partially duplicated web image retrieval.

Very Large-Scale Image Retrieval Based on Local Features.

Heterogeneous Graph Propagation for Large-Scale Web Image Search.

Indexing billions of images for sketch-based retrieval.

Hierarchical indexing scheme for fast search in a large-scale image database

Exploiting Distribution Constraints for Scalable and Efficient Image Retrieval

Spatial Coding for Large Scale Partial-Duplicate Web Image Search

Distributed Information Searching System