Abstract:Fine-grained image recognition is challenging because discriminative clues are usually fragmented, whether from a single image or multiple images. Despite their significant improvements, the majority of existing methods still focus on the most discriminative parts from a single image, ignoring informative details in other regions and lacking consideration of clues from other associated images. In this paper, we analyze the difficulties of fine-grained image recognition from a new perspective and propose a transformer architecture with the peak suppression module and knowledge guidance module, which respects the diversification of discriminative features in a single image and the aggregation of discriminative clues among multiple images. Specifically, the peak suppression module first utilizes a linear projection to convert the input image into sequential tokens. It then blocks the token based on the attention response generated by the transformer encoder. This module penalizes the attention to the most discriminative parts in the feature learning process, therefore, enhancing the information exploitation of the neglected regions. The knowledge guidance module compares the image-based representation generated from the peak suppression module with the learnable knowledge embedding set to obtain the knowledge response coefficients. Afterwards, it formalizes the knowledge learning as a classification problem using response coefficients as the classification scores. Knowledge embeddings and image-based representations are updated during training simultaneously so that the knowledge embedding includes a large number of discriminative clues for different images of the same category. Finally, we incorporate the acquired knowledge embeddings into the image-based representations as comprehensive representations, leading to significantly higher recognition performance. Extensive evaluations on the six popular datasets demonstrate the advantage of the proposed method in performance. The source code and models will be available online after the acceptance of the paper.

Knowledge Transfer and Domain Adaptation for Fine-Grained Remote Sensing Image Segmentation

A Prototypical Knowledge Oriented Adaptation Framework for Semantic Segmentation

Knowledge Based Domain Adaptation for Semantic Segmentation.

Cross-modal and Cross-domain Knowledge Transfer for Label-free 3D Segmentation

Transferring Multi-Modal Domain Knowledge to Uni-Modal Domain for Urban Scene Segmentation

Distilling Segmenters from CNNs and Transformers for Remote Sensing Images' Semantic Segmentation.

Curriculum-style Local-to-global Adaptation for Cross-domain Remote Sensing Image Segmentation

ELiTe: Efficient Image-to-LiDAR Knowledge Transfer for Semantic Segmentation

Unsupervised Domain Adaptation for Remote Sensing Semantic Segmentation with Transformer

A Multilevel-Guided Curriculum Domain Adaptation Approach to Semantic Segmentation for High-Resolution Remote Sensing Images.

Consecutive Pre-Training: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain

Avoiding Negative Transfer for Semantic Segmentation of Remote Sensing Images

TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation

Transformer with peak suppression and knowledge guidance for fine-grained image recognition

Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentation

Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation.

Domain adaptive object detection with model-agnostic knowledge transferring

Unsupervised Domain Adaptation Augmented by Mutually Boosted Attention for Semantic Segmentation of VHR Remote Sensing Images

Optimal transport-based domain adaptation for semantic segmentation of remote sensing images

Domain Adaptive Remote Sensing Scene Recognition via Semantic Relationship Knowledge Transfer