Abstract:The core of fine-grained recognition is to distinguish different subcategories within a same broad category through subtle differences in images. Yet two important factors are less explored: Firstly, the degree of similarity between categories, with the more similar categories being more likely to be confused. Secondly, different fine-grained definitions under divergent levels of expertise, with one coarse label corresponding to multiple similar fine categories (e.g., a brand of car has multiple models). In this paper, we discover that multi-granularity label can function as an intermediary for turning inter-category similarity relations into fine-grained recognition performance. Specifically, we first explore the association between label hierarchies in multi-granularity prediction: both coarse and fine label predictions require coarse information, but fine label prediction additionally requires subtle information. Then, a multi-granularity classification framework is proposed, where the extracted feature is decoupled and re-coupled into groups for predicting labels at different granularities, respectively. The key is to involve coarse-grained features in the prediction of finer-level labels. The realization way is to train these features with a joint probability-based loss that we design to reduce inter-task interference through inter-granularity probability relations. Lastly, for tasks without coarse labels, a category similarity matrix is suggested for measuring inter-class similarity and a further non-parametric aggregation method is devised for clustering fine labels into coarse labels. The evaluation is carried out on several fine-grained classification benchmark datasets. Results show that the proposed approach achieves state-of-the-art in multi-granularity classification and exhibits the potential to enhance existing fine-grained recognition models.

Coarse-to-Fine Grained Classification

Fine-Grained Visual Categorization With Fine-Tuned Segmentation

Fine-grained Image Classification Via Multi-scale Selective Hierarchical Biquadratic Pooling

Two-stage Fine-Grained Image Classification Model Based on Multi-Granularity Feature Fusion

Coarse-to-Fine Description for Fine-Grained Visual Categorization

Coarse-to-fine Image Co-segmentation with Intra and Inter Rank Constraints

Learning Enhanced Features and Inferring Twice for Fine-Grained Image Classification

Fine-grained Visual Classification Via Multilayer Bilinear Pooling with Object Localization

Fine-Grained Image Classification Based on Multi-Scale Feature Fusion

Combining Spatial Attention and Cross-Layer Bilinear Pooling for Fine-Grained Image Classification

Fine-grained Recognition: Multi-granularity Labels and Category Similarity Matrix

From Coarse to Fine: Multi-Level Feature Fusion Network for Fine-Grained Image Retrieval

Multi-Scale Feature Fusion of Covariance Pooling Networks for Fine-Grained Visual Recognition.

Multiple Granularity Descriptors for Fine-Grained Categorization.

Multi-layer Weight-Aware Bilinear Pooling for Fine-Grained Image Classification

Grouping Bilinear Pooling for Fine-Grained Image Classification

Generalized Coarse-to-Fine Visual Recognition with Progressive Training

Fine-grained Image Analysis Via Progressive Feature Learning

Cross-Layer Feature Based Multi-Granularity Visual Classification

Hierarchical Bilinear Pooling for Fine-Grained Visual Recognition

Learning Mutually Exclusive Part Representations for Fine-Grained Image Classification