Abstract:Image retrieval is crucial in robotics and computer vision, with downstream applications in robot place recognition and vision-based product recommendations. Modern retrieval systems face two key challenges: scalability and efficiency. State-of-the-art image retrieval systems train specific neural networks for each dataset, an approach that lacks scalability. Furthermore, since retrieval speed is directly proportional to embedding size, existing systems that use large embeddings lack efficiency. To tackle scalability, recent works propose using off-the-shelf foundation models. However, these models, though applicable across datasets, fall short in achieving performance comparable to that of dataset-specific models. Our key observation is that, while foundation models capture necessary subtleties for effective retrieval, the underlying distribution of their embedding space can negatively impact cosine similarity searches. We introduce Autoencoders with Strong Variance Constraints (AE-SVC), which, when used for projection, significantly improves the performance of foundation models. We provide an in-depth theoretical analysis of AE-SVC. Addressing efficiency, we introduce Single-shot Similarity Space Distillation ((SS)$_2$D), a novel approach to learn embeddings with adaptive sizes that offers a better trade-off between size and performance. We conducted extensive experiments on four retrieval datasets, including Stanford Online Products (SoP) and Pittsburgh30k, using four different off-the-shelf foundation models, including DinoV2 and CLIP. AE-SVC demonstrates up to a $16\%$ improvement in retrieval performance, while (SS)$_2$D shows a further $10\%$ improvement for smaller embedding sizes.

Image Retrieval Based on Semi-Supervised Orthogonal Discriminant Embedding

Image retrieval based on incremental subspace learning

Incremental semi-supervised subspace learning for image retrieval.

Laplacian Eigenmap for Image Retrieval

Image retrieval using dimensionality reduction

AN image retrieval method based on multiple hyperspheres OC-SVM hashing

Learning Disentangled Representation for Cross-Modal Retrieval with Deep Mutual Information Estimation.

Domain-Specific Modeling and Semantic Alignment for Image-Based 3d Model Retrieval

Semi-supervised Dimensionality Reduction for Image Retrieval

Local Feature Discriminant Projection

A Semi-supervised Orthogonal Subspace Learning with Spline Embedding

Image Retrieval Algorithm Based on Enhanced Relational Graph.

Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval.

Learning a Maximum Margin Subspace for Image Retrieval

Self-supervised Image-based 3D Model Retrieval

Locality Sensitive Semi-Supervised Feature Selection

Orthogonal Vector-Decomposed Disentanglement Network of Interactive Image Retrieval for Fashion Outfit Recommendation

Supervised Semi-Definite Embedding For Image Manifolds

Exploiting Distribution Constraints for Scalable and Efficient Image Retrieval

Image retrieval based on dimensionality reduction of second-order information

Regional Subspace Projection Coding for Image Retrieval.