Abstract:Multilabel image recognition, a critically practical task in computer vision, aims to predict multiple objects present in each image. The existing studies mainly focus on conceptual visual cues but fail to reconcile the visual information with their semantic guidance. Intuitively, humans can not only associate extra topological concepts but also imagine other approximate scenes based on a semantic description. Inspired by such semantic-interactive capability, two different types of semantic priors, i.e., the concept correlations of the same scene and semantic similarities among different scenes, should be further explored for the recognition decisions. To efficiently interact with these semantic relationships, in this article, we propose a novel semantic-interactive graph convolutional network (SI-GCN), which can leverage the topological information learned from knowledge graphs to boost the performance of multilabel recognition. Specifically, the proposed SI-GCN framework consists of two different GCN-based branches in parallel, i.e., concept correlations learning (CCL) branch and semantic similarity learning (SSL) branch. Inputting the semantic-embedding vectors of all the concepts, the CCL branch maps the label co-occurrence graph into a set of interdependent concept classifiers. Recalibrating the image feature embedding with the standardized supervision of the semantic similarity graph, the SSL branch learns the semantically consistent in-batch visual representations. Finally, a well-established interactive learning scheme is formulated to concurrently optimize the obtained concept classifiers and the visual representation learning in an end-to-end manner. Extensive experiments on the MS-COCO and Pascal VOC 2007 & 2012 benchmarks demonstrate the superiorities of the proposed SI-GCN method compared to the state-of-the-art baselines.

Multilabel Recognition Algorithm With Multigraph Structure

Multi-Label Image Recognition With Graph Convolutional Networks

Learning Graph Convolutional Networks for Multi-Label Recognition and Applications

Multi-Label Classification with Label Graph Superimposing

Multi-label Image Recognition with Two-Stream Dynamic Graph Convolution Networks.

Instance-Aware Graph Convolutional Network for Multi-Label Classification

Instance-Aware Deep Graph Learning for Multi-Label Classification

Multi-label Image Recognition with Asymmetric Co- Dependency Graphs

GKGNet: Group K-Nearest Neighbor based Graph Convolutional Network for Multi-Label Image Recognition

Graph Convolution Network Based Representation for Multi-View Multi-Label Learning

Attention-Driven Dynamic Graph Convolutional Network for Multi-label Image Recognition

A Multi-scale Semantic Attention Representation for Multi-Label Image Recognition with Graph Networks

MG-GCN: Multi-Granularity Graph Convolutional Neural Network for Multi-Label Classification in Multi-Label Information System

Fast Graph Convolution Network Based Multi-label Image Recognition via Cross-modal Fusion

Three-way Graph Convolutional Network for Multi-Label Classification in Multi-Label Information System

Multi-graph Fusion Graph Convolutional Networks with pseudo-label supervision

Multi-Label Image Classification with Attention Mechanism and Graph Convolutional Networks

Graph Attention Mechanism with Global Contextual Information for Multi-Label Image Recognition

Multigraph Fusion for Dynamic Graph Convolutional Network

Semantic-Interactive Graph Convolutional Network for Multilabel Image Recognition