Abstract:Hashing-based cross-modal search which aims to map multiple modality features into binary codes has attracted increasingly attention due to its storage and search efficiency especially in large-scale database retrieval. Recent unsupervised deep cross-modal hashing methods have shown promising results. However, existing approaches typically suffer from two limitations: (1) They usually learn cross-modal similarity information separately or in a redundant fusion manner, which may fail to capture semantic correlations among instances from different modalities sufficiently and effectively. (2) They seldom consider the sampling and weighting schemes for unsupervised cross-modal hashing, resulting in the lack of satisfactory discriminative ability in hash codes. To overcome these limitations, we propose a novel unsupervised deep cross-modal hashing method called Joint-modal Distribution-based Similarity Hashing (JDSH) for large-scale cross-modal retrieval. Firstly, we propose a novel cross-modal joint-training method by constructing a joint-modal similarity matrix to fully preserve the cross-modal semantic correlations among instances. Secondly, we propose a sampling and weighting scheme termed the Distribution-based Similarity Decision and Weighting (DSDW) method for unsupervised cross-modal hashing, which is able to generate more discriminative hash codes by pushing semantic similar instance pairs closer and pulling semantic dissimilar instance pairs apart. The experimental results demonstrate the superiority of JDSH compared with several unsupervised cross-modal hashing methods on two public datasets NUS-WIDE and MIRFlickr. Our code is available at https://github.com/KaiserLew/JDSH.

Joint and Individual Matrix Factorization Hashing for Large-Scale Cross-Modal Retrieval

Discrete Cross-Modal Hashing for Efficient Multimedia Retrieval

Efficient Discrete Supervised Hashing for Large-scale Cross-modal Retrieval

Frustratingly Easy Cross-Modal Hashing

Unsupervised Deep Hashing Via Binary Latent Factor Models for Large-scale Cross-modal Retrieval

MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal Retrieval

Large-Scale Cross-Modality Search via Collective Matrix Factorization Hashing

Joint Coupled-Hashing Representation for Cross-Modal Retrieval

Robust Supervised Matrix Factorization Hashing with Application to Cross-Modal Retrieval.

HDMFH: Hypergraph Based Discrete Matrix Factorization Hashing for Multimodal Retrieval

Collective Matrix Factorization Hashing for Multimodal Data

Supervised Matrix Factorization for Cross-Modality Hashing.

Supervised Intra- and Inter-Modality Similarity Preserving Hashing for Cross-Modal Retrieval.

Unsupervised Multi-modal Hashing for Cross-Modal Retrieval

Online supervised collective matrix factorization hashing for cross-modal retrieval

UCMH: Unpaired cross-modal hashing with matrix factorization

Online Collective Matrix Factorization Hashing for Large-Scale Cross-Media Retrieval

Cross-modal retrieval via label category supervised matrix factorization hashing

Joint-modal Distribution-based Similarity Hashing for Large-scale Unsupervised Deep Cross-modal Retrieval

Joint Specifics and Consistency Hash Learning for Large-Scale Cross-Modal Retrieval

Scalable Discriminative Discrete Hashing for Large-Scale Cross-Modal Retrieval.