Abstract:Our work focuses on tackling large-scale fine-grained image retrieval as ranking the images depicting the concept of interests (i.e., the same sub-category labels) highest based on the fine-grained details in the query. It is desirable to alleviate the challenges of both fine-grained nature of small inter-class variations with large intra-class variations and explosive growth of fine-grained data for such a practical task. In this paper, we propose attribute-aware hashing networks with self-consistency for generating attribute-aware hash codes to not only make the retrieval process efficient, but also establish explicit correspondences between hash codes and visual attributes. Specifically, based on the captured visual representations by attention, we develop an encoder-decoder structure network of a reconstruction task to unsupervisedly distill high-level attribute-specific vectors from the appearance-specific visual representations without attribute annotations. Our models are also equipped with a feature decorrelation constraint upon these attribute vectors to strengthen their representative abilities. Then, driven by preserving original entities' similarity, the required hash codes can be generated from these attribute-specific vectors and thus become attribute-aware. Furthermore, to combat simplicity bias in deep hashing, we consider the model design from the perspective of the self-consistency principle and propose to further enhance models' self-consistency by equipping an additional image reconstruction path. Comprehensive quantitative experiments under diverse empirical settings on six fine-grained retrieval datasets and two generic retrieval datasets show the superiority of our models over competing methods. Moreover, qualitative results demonstrate that not only the obtained hash codes can strongly correspond to certain kinds of crucial properties of fine-grained objects, but also our self-consistency designs can effectively overcome simplicity bias in fine-grained hashing.

Attribute-Guided Network for Cross-Modal Zero-Shot Hashing

Semantic Consistency Hashing for Cross-Modal Retrieval

COMAE: COMprehensive Attribute Exploration for Zero-shot Hashing

Cross-modal Zero-shot Hashing

Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval

PIXEL: Prompt-based Zero-shot Hashing Via Visual and Textual Semantic Alignment

Transductive Zero-Shot Hashing for Multilabel Image Retrieval

One Network for Multi-Domains: Domain Adaptive Hashing with Intersectant Generative Adversarial Networks.

Correlation Hashing Network for Efficient Cross-Modal Retrieval.

SitNet: Discrete Similarity Transfer Network for Zero-shot Hashing.

Specific class center guided deep hashing for cross-modal retrieval

Cross-Media Hashing with Neural Networks

Zero-shot Hashing Via Asymmetric Ratio Similarity Matrix

Deep Spatial Attention Hashing Network for Image Retrieval.

Attribute-Aware Deep Hashing With Self-Consistency for Large-Scale Fine-Grained Image Retrieval

A$^2$-Net: Learning Attribute-Aware Hash Codes for Large-Scale Fine-Grained Image Retrieval.

Training data ... ... 0 1 1 0 1 0 0 1 Convolution and pooling layers fc layers Hashing layer Regularized center loss Bike Car Bird Dicycle Semantic embedding loss

Deep Semantic Multimodal Hashing Network for Scalable Image-Text and Video-Text Retrievals

Deep Visual-Semantic Hashing for Cross-Modal Retrieval

Learning to Hash with Optimized Anchor Embedding for Scalable Retrieval