Abstract:Fine-grained visual categorization (FGVC), which aims at classifying objects with small inter-class variances, has been significantly advanced in recent years. However, ultra-fine-grained visual categorization (ultra-FGVC), which targets at identifying subclasses with extremely similar patterns, has not received much attention. In ultra-FGVC datasets, the samples per category are always scarce as the granularity moves down, which will lead to overfitting problems. Moreover, the difference among different categories is too subtle to distinguish even for professional experts. Motivated by these issues, this paper proposes a novel compositional feature embedding and similarity metric (CECS). Specifically, in the compositional feature embedding module, we randomly select patches in the original input image, and these patches are then replaced by patches from the images of different categories or masked out. Then the replaced and masked images are used to augment the original input images, which can provide more diverse samples and thus largely alleviate overfitting problem resulted from limited training samples. Besides, learning with diverse samples forces the model to learn not only the most discriminative features but also other informative features in remaining regions, enhancing the generalization and robustness of the model. In the compositional similarity metric module, a new similarity metric is developed to improve the classification performance by narrowing the intra-category distance and enlarging the inter-category distance. Experimental results on two ultra-FGVC datasets and one FGVC dataset with recent benchmark methods consistently demonstrate that the proposed CECS method achieves the state-of-the-art performance.

Fine-grained Image Classification by Visual-Semantic Embedding

Fine-Grained Visual Categorization With Fine-Tuned Segmentation

Learning Semantically Enhanced Feature for Fine-Grained Image Classification

Learning Enhanced Features and Inferring Twice for Fine-Grained Image Classification

Fine-Grained Image Classification Via Combining Vision And Language

Fine-Grained Image Analysis With Deep Learning: A Survey

I read, I saw, I tell: Texts Assisted Fine-Grained Visual Classification.

ELoPE: Fine-Grained Visual Classification with Efficient Localization, Pooling and Embedding

Semantic Image Segmentation Via Guidance of Image Classification

Leveraging Fine-Grained Labels to Regularize Fine-Grained Visual Classification.

Fine-Grained Visual Classification with Efficient End-to-end Localization

Granularity-aware Distillation and Structure Modeling Region Proposal Network for Fine-Grained Image Classification.

The Image Data and Backbone in Weakly Supervised Fine-Grained Visual Categorization: A Revisit and Further Thinking

HLS-FGVC: Hierarchical Label Semantics Enhanced Fine-Grained Visual Classification.

Knowledge Transfer Based Fine-Grained Visual Classification

Deep Learning for Fine-Grained Image Analysis: A Survey

A Compositional Feature Embedding and Similarity Metric for Ultra-Fine-Grained Visual Categorization

Learning Fine-Grained Semantics in Spoken Language Using Visual Grounding

Shape-guided Segmentation for Fine-Grained Visual Categorization.

Fine-Graine Visual Classification with Aggregated Object Localization and Salient Feature Suppression

Mining Discriminative Visual Features Based on Semantic Relations.