Abstract:Recently, several Vision Transformer (ViT) based methods have been proposed for Fine-Grained Visual Classification (FGVC). These methods significantly surpass existing CNN-based ones, demonstrating the effectiveness of ViT in FGVC tasks. However, there are some limitations when applying ViT directly to FGVC. First, ViT needs to split images into patches and calculate the attention of every pair, which may result in heavy noise calculation during the training phase and unsatisfying performance when handling fine-grained images with complex backgrounds and small objects. Second, complementary information is important for FGVC, but a standard ViT works by using the class token in the final layer for classification which is not enough to extract comprehensive fine-grained information at different levels. Third, the class token fuses the information of all patches in the same manner, in other words, the class token treats each patch equally. However, the discriminative parts should be more critical. To address these issues, we propose ACC-ViT including three novel components, i.e., Attention Patch Combination (APC), Critical Regions Filter (CRF), and Complementary Tokens Integration (CTI). Thereinto, APC pieces informative patches from two images to generate a new image to mitigate the noisy calculation and reinforce the differences between images. CRF emphasizes tokens corresponding to discriminative regions to generate a new class token for subtle feature learning. To extract comprehensive information, CTI integrates complementary information captured by class tokens in different ViT layers. We conduct comprehensive experiments on four widely used datasets and the results demonstrate that ACC-ViT can achieve competitive performance. The source code is available at https://github.com/Hector0426/fine-grained-image-classification-with-vit.

A free lunch from ViT:Adaptive Attention Multi-scale Fusion Transformer for Fine-grained Visual Recognition

A FREE LUNCH FROM VIT: ADAPTIVE ATTENTION MULTI-SCALE FUSION TRANSFORMER FOR FINE-GRAINED VISUAL RECOGNITION

TransFG: A Transformer Architecture for Fine-Grained Recognition

A Transformer Architecture with Adaptive Attention for Fine-Grained Visual Classification

Feature Fusion Vision Transformer for Fine-Grained Visual Categorization

RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognition

A Vision Transformer for Fine-Grained Classification by Reducing Noise and Enhancing Discriminative Information

MFF-Trans: Multi-level Feature Fusion Transformer for Fine-Grained Visual Classification

Multi-level information fusion Transformer with background filter for fine-grained image recognition

Attention-based Multi-scale ViT Fine-grained Visual Classification

AA-Trans: Core Attention Aggregating Transformer with Information Entropy Selector for Fine-grained Visual Classification

Ts-vit: feature-enhanced transformer via token selection for fine-grained image recognition

ViT-FOD: A Vision Transformer based Fine-grained Object Discriminator

MAFormer: A transformer network with multi-scale attention fusion for visual recognition

Fusion of regional and sparse attention in Vision Transformers

LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition

Constituent Attention for Vision Transformers

Recombining Vision Transformer Architecture for Fine-Grained Visual Categorization.

Facial Expression Recognition with Visual Transformers and Attentional Selective Fusion

Dual Transformer with Multi-Grained Assembly for Fine-Grained Visual Classification