Abstract:Recently, several Vision Transformer (ViT) based methods have been proposed for Fine-Grained Visual Classification (FGVC). These methods significantly surpass existing CNN-based ones, demonstrating the effectiveness of ViT in FGVC tasks. However, there are some limitations when applying ViT directly to FGVC. First, ViT needs to split images into patches and calculate the attention of every pair, which may result in heavy noise calculation during the training phase and unsatisfying performance when handling fine-grained images with complex backgrounds and small objects. Second, complementary information is important for FGVC, but a standard ViT works by using the class token in the final layer for classification which is not enough to extract comprehensive fine-grained information at different levels. Third, the class token fuses the information of all patches in the same manner, in other words, the class token treats each patch equally. However, the discriminative parts should be more critical. To address these issues, we propose ACC-ViT including three novel components, i.e., Attention Patch Combination (APC), Critical Regions Filter (CRF), and Complementary Tokens Integration (CTI). Thereinto, APC pieces informative patches from two images to generate a new image to mitigate the noisy calculation and reinforce the differences between images. CRF emphasizes tokens corresponding to discriminative regions to generate a new class token for subtle feature learning. To extract comprehensive information, CTI integrates complementary information captured by class tokens in different ViT layers. We conduct comprehensive experiments on four widely used datasets and the results demonstrate that ACC-ViT can achieve competitive performance. The source code is available at https://github.com/Hector0426/fine-grained-image-classification-with-vit.

GRVT: Toward Effective Grocery Recognition Via Vision Transformer

SLViT: Scale-Wise Language-Guided Vision Transformer for Referring Image Segmentation.

Recombining Vision Transformer Architecture for Fine-Grained Visual Categorization.

TransFG: A Transformer Architecture for Fine-Grained Recognition

Fine-Grained Grocery Product Recognition by One-Shot Learning.

A Vision Transformer for Fine-Grained Classification by Reducing Noise and Enhancing Discriminative Information

Feature Fusion Vision Transformer for Fine-Grained Visual Categorization

ZoomViT: an Observation Behavior-Based Fine-Grained Recognition Scheme

A Sequence-selective Fine-grained Image Recognition Strategy Using Vision Transformer

A free lunch from ViT:Adaptive Attention Multi-scale Fusion Transformer for Fine-grained Visual Recognition

MG-ViT: A Multi-Granularity Method for Compact and Efficient Vision Transformers

Food Recognition with Visual Transformers

LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition

CF-ViT: A General Coarse-to-Fine Method for Vision Transformer

A FREE LUNCH FROM VIT: ADAPTIVE ATTENTION MULTI-SCALE FUSION TRANSFORMER FOR FINE-GRAINED VISUAL RECOGNITION

Fine-grained image classification based on TinyVit object location and graph convolution network

Dual Guidance Enabled Fuzzy Inference for Enhanced Fine-Grained Recognition

GvT: A Graph-based Vision Transformer with Talking-Heads Utilizing Sparsity, Trained from Scratch on Small Datasets

Fine-Grained Visual Categorization with Window Restrain for Intelligent Products

A novel dual-granularity lightweight transformer for vision tasks