Abstract:Cross-modal hashing aims to map heterogeneous multimedia data into a common Hamming space through hash function, and achieves fast and flexible cross-modal retrieval. Most existing cross-modal hashing methods learn hash function by mining the correlation among multimedia data, but ignore the important property of multimedia data: Each modality of multimedia data has features of different scales, such as texture, object, and scene features in the image, which can provide complementary information for boosting retrieval task. The correlations among the multi-scale features are more abundant than the correlations between single features of multimedia data, which reveal finer underlying structures of the multimedia data and can be used for effective hashing function learning. Therefore, we propose the Multi-scale Correlation Sequential Cross-modal Hashing (MCSCH) approach, and its main contributions can be summarized as follows: (1) Multi-scale feature guided sequential hashing learning method is proposed to share the information from features of different scales through an RNN-based network and generate the hash codes sequentially. The features of different scales are used to guide the hash codes generation, which can enhance the diversity of the hash codes and weaken the influence of errors in specific features, such as false object features caused by occlusion. (2) Multi-scale correlation mining strategy is proposed to align the features of different scales in different modalities and mine the correlations among aligned features. These correlations reveal the finer underlying structure of multimedia data and can help to boost the hash function learning. (3) Correlation evaluation network evaluates the importance of the correlations to select the worthwhile correlations, and increases the impact of these correlations for hash function learning. Experiments on two widely-used 2-media datasets and a 5-media dataset demonstrate the effectiveness of our proposed MCSCH approach.

Unsupervised Cross-Media Hashing Learning Via Knowledge Graph

Supervised Coarse-to-Fine Semantic Hashing for Cross-Media Retrieval.

Online latent semantic hashing for cross-media retrieval.

Discrete Cross-Modal Hashing for Efficient Multimedia Retrieval

Unsupervised Cross-Media Hashing with Structure Preservation

Unsupervised Multi-modal Hashing for Cross-Modal Retrieval

Cross-Media Hashing with Neural Networks

Scalable Multimedia Retrieval By Deep Learning Hashing With Relative Similarity Learning

Cross-media hashing with kernel regression

Robust Unsupervised Cross-modal Hashing for Multimedia Retrieval

Dark knowledge association guided hashing for unsupervised cross-modal retrieval

Unsupervised Generative Adversarial Cross-Modal Hashing

Aggregation-Based Graph Convolutional Hashing for Unsupervised Cross-Modal Retrieval

Efficient Supervised Graph Embedding Hashing for large-scale cross-media retrieval

Supervised Semantic-Embedded Hashing for Multimedia Retrieval

Joint Image-Text Hashing for Fast Large-Scale Cross-Media Retrieval Using Self-Supervised Deep Learning.

Unsupervised Deep K-Means Hashing for Efficient Image Retrieval and Clustering

Sequential Cross-Modal Hashing Learning Via Multi-scale Correlation Mining

Unsupervised Cross-modal Hashing via Semantic Text Mining