Abstract:Multi-label image recognition aims to predict a set of labels that present in an image. The key to deal with such problem is to mine the associations between image contents and labels, and further obtain the correct assignments between images and their labels. In this paper, we treat each image as a bag of instances, and formulate the task of multi-label image recognition as an instance-label matching selection problem. To model such problem, we propose an innovative Semantic-aware Graph Matching framework for Multi-Label image recognition (ML-SGM), in which Graph Matching mechanism is introduced owing to its good performance of excavating the instance and label relationship. The framework explicitly establishes category correlations and instance-label correspondences by modeling the relation among content-aware (instance) and semantic-aware (label) category representations, to facilitate multi-label image understanding and reduce the dependency of large amounts of training samples for each category. Specifically, we first construct an instance spatial graph and a label semantic graph respectively and then incorporate them into a constructed assignment graph by connecting each instance to all labels. Subsequently, the graph network block is adopted to aggregate and update all nodes and edges state on the assignment graph to form structured representations for each instance and label. Our network finally derives a prediction score for each instance-label correspondence and optimizes such correspondence with a weighted cross-entropy loss. Empirical results conducted on generic multi-label image recognition demonstrate the superiority of our proposed method. Moreover, the proposed method also shows advantages in multi-label recognition with partial labels and multi-label few-shot learning, as well as outperforms current state-of-the-art methods with a clear margin. Our code is available at https://github.com/yananwu0510/ML-SGM.

Beyond Word Embeddings: Heterogeneous Prior Knowledge Driven Multi-Label Image Classification

A multi-label image classification method combining multi-stage image semantic information and label relevance

Dual Collaborative Visual-Semantic Mapping for Multi-Label Zero-Shot Image Recognition

GM-MLIC: Graph Matching Based Multi-Label Image Classification

Semantic-Interactive Graph Convolutional Network for Multilabel Image Recognition

MULTI-LABEL IMAGE RECOGNITION WITH JOINT CLASS-AWARE MAP DISENTANGLING AND LABEL CORRELATION EMBEDDING

Learning Graph Convolutional Networks for Multi-Label Recognition and Applications

Multi-Label Classification with Label Graph Superimposing

Semantic and Correlation Disentangled Graph Convolutions for Multilabel Image Recognition.

Multi-Label Image Recognition With Graph Convolutional Networks

AdaHGNN: Adaptive Hypergraph Neural Networks for Multi-Label Image Classification

Semantic Supplementary Network With Prior Information for Multi-Label Image Classification

Active Learning in Multi-Label Image Classification with Graph Convolutional Network Embedding

Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attention

Semantic-Aware Graph Matching Mechanism for Multi-Label Image Recognition

MLGN:A Multi-Label Guided Network for Improving Text Classification

Joint multi-label multi-instance learning for image classification

Multi-Label Dictionary Learning for Image Annotation

A New multi-instance multi-label learning approach for image and text classification

Multi-Manifold Sparse Graph Embedding for Multi-Modal Image Classification

Semantic Embedded Deep Neural Network: A Generic Approach to Boost Multi-Label Image Classification Performance