Abstract:Recently, remarkable progress on multi-label image classification has been achieved by locating semantic-agnostic image regions and extracting their features with deep convolutional neural networks. However, existing pipelines depend on the hypothesis region generation step, which typically brings about extra computational costs, e.g., generating hundreds of meaningless proposals and extracting their features. Moreover, the contextual dependencies among these localized regions are usually ignored or oversimplified during the learning and inference stages. To resolve these issues, we develop a novel attentive transformer-localizer (ATL) module that contains differential transformations (e.g., translation, scale), which can automatically discover the discriminative semantic-aware regions from input images in terms of multi-label recognition. This module can be flexibly incorporated with recurrent neural networks such as the long short-term memory (LSTM) network for memorizing and updating the contextual dependencies of the localized regions. We thus build a unified multi-label image recognition framework. Specifically, the ATL module is applied to progressively localize the attentive regions from the convolutional feature maps in a proposal-free manner, and the LSTM network sequentially predicts label scores for the localized regions and updates the parameters of the ATL module while capturing the global dependencies among these regions. To associate the localized regions with semantic labels over diverse locations and scales, we further design three constraints together with the ATL module. Extensive experiments and evaluations on two large-scale benchmarks (i.e., PASCAL VOC and Microsoft COCO) show that the proposed approach achieves superior performance over existing state-of-the-art methods in terms of both performance and efficiency.

Order-Free RNN With Visual Attention for Multi-Label Classification

Visual Attention in Multi-Label Image Classification.

Multi-Label Image Classification with Attention Mechanism and Graph Convolutional Networks

Residual Attention: A Simple but Effective Method for Multi-Label Recognition

Order-Free Learning Alleviating Exposure Bias in Multi-Label Classification.

Spatial Context-Aware Object-Attentional Network for Multi-Label Image Classification

Double Attention Based on Graph Attention Network for Image Multi-Label Classification

Relation Network for Multi-label Aerial Image Classification

Label-Guided Cross-Modal Attention Network for Multi-Label Aerial Image Classification

Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks.

Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition

A Novel Multi-Attention Driven System for Multi-Label Remote Sensing Image Classification

A Label-Specific Attention-Based Network with Regularized Loss for Multi-label Classification

ASRNN: A recurrent neural network with an attention model for sequence labeling

Learning Attentional Recurrent Neural Network for Visual Tracking

Research of multi-label text classification based on label attention and correlation networks

Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning

Where to Look: A Unified Attention Model for Visual Recognition with Reinforcement Learning

Multi-label Image Recognition with Attentive Transformer-Localizer Module

Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition

Multi-Label Remote Sensing Image Scene Classification by Combining a Convolutional Neural Network and a Graph Neural Network