Abstract:Recently, numerous unsupervised cross-modal hashing methods have been proposed to deal the image-text retrieval tasks for the unlabeled cross-modal data. However, when these methods learn to generate hash codes, almost all of them lack modality-interaction in the following two aspects: (1) The instance similarity matrix used to guide the hashing networks training is constructed without image-text interaction, which fails to capture the fine-grained cross-modal cues to elaborately characterize the intrinsic semantic similarity among the datapoints. (2) The binary codes used for quantization loss are inferior because they are generated by directly quantizing a simple combination of continuous hash codes from different modalities without the interaction among these continuous hash codes. Such problems will cause the generated hash codes to be of poor quality and degrade the retrieval performance. Hence, in this paper, we propose a novel Unsupervised Cross-modal Hashing with Modality-interaction, termed UCHM. Specifically, by optimizing a novel hash-similarity-friendly loss, a modality-interaction-enabled (MIE) similarity generator is first trained to generate a superior MIE similarity matrix for the training set. Then, the generated MIE similarity matrix is utilized as guiding information to train the deep hashing networks. Furthermore, during the process of training the hashing networks, a novel bit-selection module is proposed to generate high-quality unified binary codes for the quantization loss with the interaction among continuous codes from different modalities, thereby further enhancing the retrieval performance. Extensive experiments on two widely used datasets show that the proposed UCHM outperforms state-of-the-art techniques on cross-modal retrieval tasks.

UCMH: Unpaired cross-modal hashing with matrix factorization

Discrete Cross-Modal Hashing for Efficient Multimedia Retrieval

Frustratingly Easy Cross-Modal Hashing

Discrete Similarity Preserving Hashing for Cross-modal Retrieval.

Semantic Consistency Hashing for Cross-Modal Retrieval

Robust Unsupervised Cross-modal Hashing for Multimedia Retrieval

Large-Scale Cross-Modality Search via Collective Matrix Factorization Hashing

Completely Unpaired Cross-Modal Hashing Based on Coupled Subspace

Unsupervised Multi-modal Hashing for Cross-Modal Retrieval

Collective Matrix Factorization Hashing for Multimodal Data

Asymmetric Supervised Consistent and Specific Hashing for Cross-Modal Retrieval

Adaptive Marginalized Semantic Hashing for Unpaired Cross-Modal Retrieval

MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal Retrieval

Unsupervised Deep Hashing Via Binary Latent Factor Models for Large-scale Cross-modal Retrieval

Deep Unified Cross-Modality Hashing by Pairwise Data Alignment

Unsupervised Cross-modal Hashing with Modality-interaction

Fast Unmediated Hashing for Cross-Modal Retrieval.

Unsupervised Online Cross-modal Hashing with Multiple Association Exploitation

Unsupervised Cross-modal Hashing via Semantic Text Mining