Abstract:Exploiting relationship among samples in cross-modal data plays a key role in the task of cross-modal retrieval, but most of existing methods only extract the correlation from pairwise samples and ignore the relations of unpaired samples. Some graph regularization methods proposed a reasonable paradigm to exploit the correlation from multiple samples. However, limited by the traditional framework, the performance has much room to improve. Moreover, although some existing DNN-based methods achieve excellent performance, the requirement of massive labeled data is also a shortcoming. In this paper, we propose a novel semi-supervised method, named Semi-supervised Constrained Graph Convolutional Network (SCGCN), which adopts graph convolutional network to exploit correlation from batch samples of data with different modalities. For reducing the requirement of labeled data, we design a two stage training procedure: deep supervised learning stage and unsupervised learning stage. In deep supervised learning stage, we integrate two DNN-based semantic encoding networks and a shared classifier into Deep Cross-modal Semantic Encoding (DCSE) module which is trained by supervised learning with labeled data. From DCSE module, we learn a temporary modality-invariant space where the semantic embeddings of samples with different modalities are modality-invariant, and we also learn a classifier which can generate predicted label from the unlabeled data. In unsupervised learning stage, for fully exploiting the correlation from cross-modal data, we design a Constrained Graph Convolutional Network (CGCN) module which utilizes GCN to exploit the correlation and adopts both intra-modal discriminative loss and inter-modal pairwise similar loss to ensure the generated common representation modality-invariant and semantical discriminative. We perform extensive experiments on four conventional datasets and a large scale dataset to demonstrate the effectiveness of proposed approach.

Semi-supervised Learning Remote Sensing Image Retrieval MethodBased on Triplet Sampling Graph Convolutional Network br

A Novel Metric Learning Method Based On The Triplet Sampling Graph Convolutional Network For Remote Sensing Image Retrieval

Enhancing Remote Sensing Image Retrieval with Triplet Deep Metric Learning Network

A Three-Layered Graph-Based Learning Approach for Remote Sensing Image Retrieval.

Deep Graph Convolutional Quantization Networks for Image Retrieval.

Feature Extraction with Triplet Convolutional Neural Network for Content-Based Image Retrieval

Deep Metric Learning Based on Scalable Neighborhood Components for Remote Sensing Scene Characterization

Supervised Contrastive Learning Based on Fusion of Global and Local Features for Remote Sensing Image Retrieval

Image retrieval for Structure-from-Motion via Graph Convolutional Network

A Novel Graph-Theoretic Deep Representation Learning Method for Multi-Label Remote Sensing Image Retrieval

Context Spatial Awareness Remote Sensing Image Change Detection Network Based on Graph and Convolution Interaction

Spectral-Spatial Graph Convolutional Networks for Semel-Supervised Hyperspectral Image Classification

An Unsupervised Remote Sensing Change Detection Method Based on Multiscale Graph Convolutional Network and Metric Learning

Semi-supervised constrained graph convolutional network for cross-modal retrieval

Non-Local Similarity-Based Attentive Graph Convolution Network for Remote Sensing Image Super-Resolution

Content-Based High-Resolution Remote Sensing Image Retrieval Via Unsupervised Feature Learning And Collaborative Affinity Metric Fusion

Text-Image Matching for Cross-Modal Remote Sensing Image Retrieval via Graph Neural Network

Semi-Supervised Tree Species Classification for Multi-Source Remote Sensing Images Based on a Graph Convolutional Neural Network

Dynamic manifold-based sample selection in contrastive learning for remote sensing image retrieval

Hi-SIGIR: Hierachical Semantic-Guided Image-to-image Retrieval Via Scene Graph

Graph-based Multimodal Semi-Supervised Image Classification