Abstract:Fine-grained image recognition is a challenging task that focuses on identifying images from similar subordinate categories. Recently, methods based on the vision transformer (ViT) have demonstrated remarkable achievements in fine-grained image recognition, which inherent multi-head self-attention (MHSA) can effectively capture the discriminative regions in images. However, most of these ViT-based methods ignore the channel relationships of image features, and there are also problems with the inconsistent learning performance of different heads in MHSA and different layers in ViT. To address these issues, an innovative feature-enhanced transformer is proposed, named TS-ViT. TS-ViT includes three key modules: soft channel attention (Soft-CA), multi-head token selection (MHTS), and multi-level feature enhancement (MLFE). The Soft-CA enables the model to concentrate on the relationships among various channels of image features. The MHTS is proposed to address the issue of inconsistent multi-head learning performance. It selects tokens with discriminative region positions based on attention maps to form the multi-level feature. By employing contrastive learning and enhanced feature extraction, the MLFE is proposed to effectively utilize multi-level features while mitigating background noise. Extensive experiments have demonstrated that TS-ViT achieves superior performance compared to popular methods, with average accuracy of 91.8%, 91.2%, 99.5%, and 93.9% on the experimental data sets, respectively. Furthermore, TS-ViT demonstrated outstanding performance in terms of computational complexity and efficiency, with an average parameter count of 93.5M, FLOPs of 73.2G, training time of 7.8 h, and inference time of 4.1 milliseconds.

HAT: A Visual Transformer Model for Image Recognition Based on Hierarchical Attention Transformation

Scalable Vision Transformers with Hierarchical Pooling.

Vision Transformers with Hierarchical Attention

FasterViT: Fast Vision Transformers with Hierarchical Attention

HAT: Hybrid Attention Transformer for Image Restoration

DctViT: Discrete Cosine Transform Meet Vision Transformers

Hierarchical Attention Transformer for Hyperspectral Image Classification

CAT: Cross Attention in Vision Transformer

Convolutional Embedding Makes Hierarchical Vision Transformer Stronger

DAT++: Spatially Dynamic Vision Transformer with Deformable Attention

Ts-vit: feature-enhanced transformer via token selection for fine-grained image recognition

What Makes for Hierarchical Vision Transformer?

Improving Vision Transformers by Revisiting High-Frequency Components

FDViT: Improve the Hierarchical Architecture of Vision Transformer.

HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling

Rethinking Attention Mechanisms in Vision Transformers with Graph Structures

ASAFormer: Visual tracking with convolutional vision transformer and asymmetric selective attention

Data Augmentation Vision Transformer for Fine-grained Image Classification

ReViT: Enhancing Vision Transformers Feature Diversity with Attention Residual Connections

Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning

Nested-TNT: Hierarchical Vision Transformers with Multi-Scale Feature Processing