Abstract:Cross-modal Hashing (CMH) retrieval aims to mutually search data from heterogeneous modalities by projecting original modality data into a common hamming space, with the significant advantages of low storage and computing costs. However, CMH remains challenging for multi-label cross-modal datasets. Firstly, preserving content similarity would inevitably be deficient under the representation of short-length binary codes. Secondly, different semantics are treated independently, whereas their co-occurrences are neglected, reducing retrieval quality. Thirdly, the commonly used metric learning objective is ineffective in capturing similarity information at a fine-grained level, leading to the imprecise preservation of such information. Therefore, we propose a Deep Cross-Modal Hashing with Multi-Task Latent Space Learning (DMLSH) framework to tackle these bottlenecks. For a more thorough excavation of distinctive features with diverse characteristics underneath heterogeneous data, our DMLSH is designed to preserve three different types of knowledge. The first is the semantic relevance and co-occurrence with the integration of the attention module and the Long Short-Term Memory (LSTM) layer; The second is the highly precise pairwise correlation considering the quantification of semantic similarity with self-paced optimization; The last is the pairwise similarity information discovered by a self-supervised semantic network from a perspective of probabilistic knowledge transfer. Abundant knowledge from the latent spaces is seamlessly refined and fused into a common Hamming space by a hashing attention mechanism, facilitating the discrimination of hash codes and the elimination of modalities’ heterogeneity. Exhaustive experiments demonstrate the state-of-the-art performance of our proposed DMLSH on four mainstream cross-modal retrieval benchmarks.

Deep Multi-Similarity Hashing for Multi-label Image Retrieval

Deep Multi-Similarity Hashing with semantic-aware preservation for multi-label image retrieval

Deep Multi-Similarity Hashing Via Label-Guided Network for Cross-Modal Retrieval

Deep Supervised Hashing for Multi-Label and Large-Scale Image Retrieval

Deep Semantic Ranking Based Hashing for Multi-Label Image Retrieval

Deep Multi-Label Hashing For Large-Scale Visual Search Based On Semantic Graph

Deep Top Similarity Hashing with Class-Wise Loss for Multi-Label Image Retrieval

Deep Hashing with Multi-Central Ranking Loss for Multi-Label Image Retrieval.

Instance Similarity Deep Hashing for Multi-Label Image Retrieval.

Learning Multilevel Semantic Similarity for Large-Scale Multi-Label Image Retrieval.

Improved Deep Hashing with Soft Pairwise Similarity for Multi-Label Image Retrieval.

Multi-task Learning for Deep Semantic Hashing

Instance-Aware Hashing for Multi-Label Image Retrieval.

Image Retrieval via Balanced and Maximum Variance Deep Hashing.

Deep Multi-Level Semantic Hashing for Cross-Modal Retrieval

Multi-Label Deep Sparse Hashing

Deep Multi-Negative Supervised Hashing for Large-Scale Image Retrieval

Scalable Multimedia Retrieval By Deep Learning Hashing With Relative Similarity Learning

DSRPH: Deep Semantic-Aware Ranking Preserving Hashing for Efficient Multi-Label Image Retrieval

Deep Cross-Modal Hashing with Multi-Task Latent Space Learning

Multi-label Contrastive Hashing