Abstract:Composed image retrieval attempts to retrieve an image of interest from gallery images through a composed query of a reference image and its corresponding modified text. It has recently attracted attention due to the collaboration of information-rich images and concise language to precisely express the requirements of target images. Most current composed image retrieval methods follow a supervised learning approach to training on a costly triplet dataset composed of a reference image, modified text, and a corresponding target image. To avoid difficult to-obtain labeled triplet training data, zero-shot composed image retrieval (ZS-CIR) has been introduced, which aims to retrieve the target image by learning from image-text pairs (self-supervised triplets), without the need for human-labeled triplets. However, this self-supervised triplet learning approach is computationally less effective and less understandable as it assumes the interaction between image and text is conducted with implicit query embedding without explicit semantical interpretation. In this work, we present a new training-free zero-shot composed image retrieval method which translates the query into explicit human-understandable text. This helps improve model learning efficiency to enhance the generalization capacity of foundation models. Further, we introduce a Local Concept Re-ranking (LCR) mechanism to focus on discriminative local information extracted from the modified instructions. Extensive experiments on four ZS-CIR benchmarks show that our method achieves comparable performances to that of the state of-the-art triplet training based methods, but significantly outperforms other training-free methods on the open domain datasets (CIRR, CIRCO and COCO), as well as the fashion domain dataset (FashionIQ).

Composed Query Image Retrieval Using Locally Bounded Features

Multimodal Composition Example Mining for Composed Query Image Retrieval

Composition Analysis-Based Relevance Ranking for Ancient Mural

Composition Based Semantic Scene Retrieval For Ancient Murals

Viewpoint-Aware Representation for Sketch-Based 3D Model Retrieval

Cross-Modal Joint Prediction and Alignment for Composed Query Image Retrieval

Multi-order adversarial representation learning for composed query image retrieval

Geometry Sensitive Cross-Modal Reasoning for Composed Query Based Image Retrieval

Composed Image Retrieval for Remote Sensing

Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image Retrieval

Color Image Retrieval Utilizing Extended Fast Vq Codeword Search Technique And Vector Composition-Based Feedback

Multi-Grained Attention Network with Mutual Exclusion for Composed Query-Based Image Retrieval

Target-Guided Composed Image Retrieval

Flickr30K-CFQ: A Compact and Fragmented Query Dataset for Text-image Retrieval

Semantic Distillation from Neighborhood for Composed Image Retrieval

Composed Image Retrieval for Training-Free Domain Conversion

Multi-Query Parallel Field Ranking for image retrieval.

Enhancing Historical Image Retrieval with Compositional Cues

Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features

Training-free Zero-shot Composed Image Retrieval with Local Concept Reranking

Multi-view and region reasoning semantic enhancement for image-text retrieval