Abstract:Multilabel image recognition, a critically practical task in computer vision, aims to predict multiple objects present in each image. The existing studies mainly focus on conceptual visual cues but fail to reconcile the visual information with their semantic guidance. Intuitively, humans can not only associate extra topological concepts but also imagine other approximate scenes based on a semantic description. Inspired by such semantic-interactive capability, two different types of semantic priors, i.e., the concept correlations of the same scene and semantic similarities among different scenes, should be further explored for the recognition decisions. To efficiently interact with these semantic relationships, in this article, we propose a novel semantic-interactive graph convolutional network (SI-GCN), which can leverage the topological information learned from knowledge graphs to boost the performance of multilabel recognition. Specifically, the proposed SI-GCN framework consists of two different GCN-based branches in parallel, i.e., concept correlations learning (CCL) branch and semantic similarity learning (SSL) branch. Inputting the semantic-embedding vectors of all the concepts, the CCL branch maps the label co-occurrence graph into a set of interdependent concept classifiers. Recalibrating the image feature embedding with the standardized supervision of the semantic similarity graph, the SSL branch learns the semantically consistent in-batch visual representations. Finally, a well-established interactive learning scheme is formulated to concurrently optimize the obtained concept classifiers and the visual representation learning in an end-to-end manner. Extensive experiments on the MS-COCO and Pascal VOC 2007 & 2012 benchmarks demonstrate the superiorities of the proposed SI-GCN method compared to the state-of-the-art baselines.

Semi-supervised Image Annotation with Parallel Graph Convolutional Networks

Parallel Hypergraph Convolutional Neural Networks for Image Annotation

Feature Fusion Based Parallel Graph Convolutional Neural Network for Image Annotation

Collaborative Graph Convolutional Networks: Unsupervised Learning Meets Semi-Supervised Learning.

Central Attention with Multi-Graphs for Image Annotation

Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning

Image Annotation Based on Semantic Structure and Graph Learning

Semi-Supervised Learning with Graph Learning-Convolutional Networks.

Deep Graph-Convolutional Generative Adversarial Network for Semi-Supervised Learning on Graphs

Image Annotation by K NN-sparse Graph-Based Label Propagation over Noisily Tagged Web Images

A Safe Semi-supervised Graph Convolution Network

Multi-label Image Recognition with Two-Stream Dynamic Graph Convolution Networks.

Semi-supervised Learning with Adaptive Neighborhood Graph Propagation Network

Class-related Graph Convolution for Weakly Supervised Semantic Segmentation

GraphNet: Learning Image Pseudo Annotations for Weakly-Supervised Semantic Segmentation.

Self-consistent Graph Neural Networks for Semi-supervised Node Classification

Self-SAGCN: Self-Supervised Semantic Alignment for Graph Convolution Network

Semantic-Interactive Graph Convolutional Network for Multilabel Image Recognition

Learnable Graph Convolutional Network with Semisupervised Graph Information Bottleneck.

Expanding Training Set for Graph-Based Semi-supervised Classification.

CSGNN: Improving Graph Neural Networks with Contrastive Semi-supervised Learning