Abstract:This paper presents a novel keypoints-based attention mechanism for visual recognition in still images. Deep Convolutional Neural Networks (CNNs) for recognizing images with distinctive classes have shown great success, but their performance in discriminating fine-grained changes is not at the same level. We address this by proposing an end-to-end CNN model, which learns meaningful features linking fine-grained changes using our novel attention mechanism. It captures the spatial structures in images by identifying semantic regions (SRs) and their spatial distributions, and is proved to be the key to modelling subtle changes in images. We automatically identify these SRs by grouping the detected keypoints in a given image. The ``usefulness'' of these SRs for image recognition is measured using our innovative attentional mechanism focusing on parts of the image that are most relevant to a given task. This framework applies to traditional and fine-grained image recognition tasks and does not require manually annotated regions (e.g. bounding-box of body parts, objects, etc.) for learning and prediction. Moreover, the proposed keypoints-driven attention mechanism can be easily integrated into the existing CNN models. The framework is evaluated on six diverse benchmark datasets. The model outperforms the state-of-the-art approaches by a considerable margin using Distracted Driver V1 (Acc: 3.39%), Distracted Driver V2 (Acc: 6.58%), Stanford-40 Actions (mAP: 2.15%), People Playing Musical Instruments (mAP: 16.05%), Food-101 (Acc: 6.30%) and Caltech-256 (Acc: 2.59%) datasets.

Learning Scale-Consistent Attention Part Network for Fine-grained Image Recognition

Attention cutting and padding learning for fine-grained image recognition

Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition

A Multi-part Convolutional Attention Network for Fine-Grained Image Recognition

Learning Rich Part Hierarchies with Progressive Attention Networks for Fine-Grained Image Recognition

Fully Convolutional Attention Networks for Fine-Grained Recognition

Scale Channel Attention Network for Image Segmentation

Attention-based cropping and erasing learning with coarse-to-fine refinement for fine-grained visual classification

Spatial-Scale Aligned Network for Fine-Grained Recognition

Discriminative Region Enhancing and Suppression Network for Fine-Grained Visual Categorization

Part Attentioned Fine-Grained Categorization Using Context Sensitive Flow

Attend and Guide (AG-Net): A Keypoints-driven Attention-based Deep Network for Image Recognition

Research on image classification based on residual group multi-scale enhanced attention network

LOCAL TO GLOBAL WITH MULTI-SCALE ATTENTION NETWORK FOR PERSON RE-IDENTIFICATION

ScaleNAS: One-Shot Learning of Scale-Aware Representations for Visual Recognition

Dual attention guided multi-scale CNN for fine-grained image classification

Multi-branch and Multi-scale Attention Learning for Fine-Grained Visual Categorization

Weakly supervised fine-grained recognition based on spatial-channel aware attention filters

Selective Sparse Sampling for Fine-Grained Image Recognition

SCA-Net: A Multiscale Building Segmentation Network Incorporating a Dual-Attention Mechanism

AutoScaler: Scale-Attention Networks for Visual Correspondence