Abstract:The core of cross-modal hashing methods is to map high dimensional features into binary hash codes, which can then efficiently utilize the Hamming distance metric to enhance retrieval efficiency. Recent development emphasizes the advantages of the unsupervised cross-modal hashing technique, since it only relies on relevant information of the paired data, making it more applicable to real-world applications. However, two problems, that is intro-modality correlation and inter-modality correlation, still have not been fully considered. Intra-modality correlation describes the complex overall concept of a single modality and provides semantic relevance for retrieval tasks, while inter-modality correction refers to the relationship between different modalities. From our observation and hypothesis, the dependency relationship within the modality and between different modalities can be constructed at the object level, which can further improve cross-modal hashing retrieval accuracy. To this end, we propose a Visual-textful Correlation Graph Hashing (OVCGH) approach to mine the fine-grained object-level similarity in cross-modal data while suppressing noise interference. Specifically, a novel intra-modality correlation graph is designed to learn graph-level representations of different modalities, obtaining the dependency relationship of the image region to image region and the tag to tag in an unsupervised manner. Then, we design a visual-text dependency building module that can capture correlation semantic information between different modalities by modeling the dependency relationship between image object region and text tag. Extensive experiments on two widely used datasets verify the effectiveness of our proposed approach.

Visual Words Assignment on A Graph Via Minimal Mutual Information Loss

Visual words assignment via information-theoretic manifold embedding.

Learning Visually Aligned Semantic Graph for Cross-Modal Manifold Matching.

Nonlinear Discrete Cross-Modal Hashing for Visual-Textual Data

Visual-Semantic Graph Matching for Visual Grounding

Visual word coding based on difference maximization.

Learning to Infer Unseen Single-/ Multi-Attribute-Object Compositions with Graph Networks.

Large Visual Words For Large Scale Image Classification

Randomized Locality Sensitive Vocabularies For Bag-Of-Features Model

Visual Words Refining Exploiting Spatial Co-Occurrence Table

High-Order Topology Modeling of Visual Words for Image Classification

Fast Visual Word Quantization Via Spatial Neighborhood Boosting

Visual language modeling for image classification.

Object-Level Visual-Text Correlation Graph Hashing for Unsupervised Cross-Modal Retrieval

Learning Compact Binary Codes Via Pairwise Correlation Reconstruction

Unifying Discriminative Visual Codebook Generation with Classifier Training for Object Category Recognition

Latent Visual Context Learning for Web Image Applications

Graph Embedding Learning for Cross-Modal Information Retrieval.

Structurally Enhanced Incremental Neural Learning for Image Classification with Subgraph Extraction

Building Descriptive and Discriminative Visual Codebook for Large-Scale Image Applications.

Visual Feature Coding for Image Classification Integrating Dictionary Structure