Abstract:Multi-label image classification is a fundamental yet challenging task, which aims to predict the labels associated with a given image. Most of previous methods directly exploit the high-level features from the last layer of convolutional neural network for classification. However, these methods cannot obtain global features due to the limited size of convolutional kernels, and they fail to extract multi-scale features to effectively recognize small-scale objects in the images. Recent studies exploit the graph convolution network to model the label correlations for boosting the classification performance. Despite substantial progress, these methods rely on manually pre-defined graph structures. Besides, they ignore the associations between semantic labels and image regions, and do not fully explore the spatial context of images. To address above issues, we propose a novel Dual Attention Transformer (DATran) model, which adopts a dual-stream architecture that simultaneously learns spatial and channel correlations from multi-label images. Firstly, in order to solve the problem that current methods are difficult to recognize small-size objects, we develop a new multi-scale feature fusion (MSFF) module to generate multi-scale feature representation by jointly integrating both high-level semantics and low-level details. Secondly, we design a prior-enhanced spatial attention (PSA) module to learn the long-range correlation between objects from different spatial positions in images to enhance the model performance. Thirdly, we devise a prior-enhanced channel attention (PCA) module to capture the inter-dependencies between different channel maps, thus effectively improving the correlation between semantic categories. It is worth noting that PSA module and PCA module complement and promote each other to further augment the feature representations. Finally, the outputs of these two attention modules are fused to obtain the final features for classification. Performance evaluation experiments are conducted on MS-COCO 2014, PASCAL VOC 2007 and VG-500 datasets, demonstrating that DATran model achieves better performance than current state-of-the-art models.

Multi-scale and Discriminative Part Detectors Based Features for Multi-label Image Classification.

Hcp: A Flexible Cnn Framework for Multi-Label Image Classification

Feature Enhancement for Multi-scale Object Detection.

Exploit Bounding Box Annotations for Multi-Label Object Recognition

CNN: Single-label to Multi-label

Coarse-to-fine Trained Multi-Scale Convolutional Neural Networks for Image Classification.

DATran: Dual Attention Transformer for Multi-Label Image Classification

Using Multi-Label Classification to Improve Object Detection.

Combining Local and Global Hypotheses in Deep Neural Network for Multi-Label Image Classification

Multilabel Convolutional Network With Feature Denoising and Details Supplement

A Multi-Feature Fusion Model Based on Denoising Convolutional Neural Network and Attention Mechanism for Image Classification

Multi-Channel and Multi-Scale Mid-Level Image Representation for Scene Classification

A Multi-part Convolutional Attention Network for Fine-Grained Image Recognition

Integrating hierarchical feature selection and classifier training for multi-label image annotation.

An Attention-Driven Multi-label Image Classification with Semantic Embedding and Graph Convolutional Networks.

Multi-Scale Feature Fusion of Covariance Pooling Networks for Fine-Grained Visual Recognition.

MCDet: Multi-Content Collaboration Detector for Multiscale Remote Sensing Object

Multi-scale Vertical Cross-layer Feature Aggregation and Attention Fusion Network for Object Detection

Scene Understanding Based on Multi-Scale Pooling of Deep Learning Features

Improve Object Detection Via a Multi-Feature and Multi-Task CNN Model

Learning Discriminated Features Based on Feature Pyramid Networks and Attention for Multi-scale Object Detection