Abstract:The task of image multi-label classification is to accurately recognize multiple objects in an input image. Most of the recent works need to leverage the label co-occurrence matrix counted from training data to construct the graph structure, which are inflexible and may degrade model generalizability. In addition, these methods fail to capture the semantic correlation between the channel feature maps to further improve model performance. To address these issues, we propose DA-GAT (a D ouble A ttention framework based on the G raph A ttention ne T work) to effectively learn the correlation between labels from training data. First, we devise a new channel attention mechanism to enhance the semantic correlation between channel feature maps, so as to implicitly capture the correlation between labels. Second, we propose a new label attention mechanism to avoid the adverse impact of a manually constructed label co-occurrence matrix. It only needs to leverage the label embedding as the input of network, then automatically constructs the label relation matrix to explicitly establish the correlation between labels. Finally, we effectively fuse the output of these two attention mechanisms to further improve model performance. Extensive experiments are conducted on three public multi-label classification benchmarks. Our DA-GAT model achieves mean average precision of 87.1%, 96.6%, and 64.3% on MS-COCO 2014, PASCAL VOC 2007, and NUS-WIDE, respectively, and obviously outperforms other existing state-of-the-art methods. In addition, visual analysis experiments demonstrate that each attention mechanism can capture the correlation between labels well and significantly promote the model performance.

G-CAM: Graph Convolution Network Based Class Activation Mapping for Multi-label Image Recognition.

Exclusive Feature Constrained Class Activation Mapping for Better Visual Explanation.

Learning Graph Convolutional Networks for Multi-Label Recognition and Applications

Label-Guided Cross-Modal Attention Network for Multi-Label Aerial Image Classification

GT-CAM: Game Theory Based Class Activation Map for GCN

Multi-Label Image Recognition With Graph Convolutional Networks

Master-CAM: Multi-scale fusion guided by Master map for high-quality class activation maps

Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition

Multilabel Recognition Algorithm With Multigraph Structure

Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition

Cluster-CAM: Cluster-weighted visual interpretation of CNNs' decision in image classification

Extracting Class Activation Maps from Non-Discriminative Features as well

HAM: Hybrid Attention Module in Deep Convolutional Neural Networks for Image Classification

CAM-loss: Towards Learning Spatially Discriminative Feature Representations

Double Attention Based on Graph Attention Network for Image Multi-Label Classification

FD-CAM: Improving Faithfulness and Discriminability of Visual Explanation for CNNs

CR-CAM: Generating explanations for deep neural networks by contrasting and ranking features

Cross-modal Fusion for Multi-Label Image Classification with Attention Mechanism

Gaze-Guided Class Activation Mapping: Leveraging Human Attention for Network Attention in Chest X-rays Classification

Adapting Grad-CAM for Embedding Networks