Abstract:Fine-grained visual categorization (FGVC) aims to discriminate similar subcategories that belong to the same superclass. Since the distinctions among similar subcategories are quite subtle and local, it is highly challenging to distinguish them from each other even for humans. So the localization of distinctions is essential for fine-grained visual categorization, and there are two pivotal problems: (1) Which regions are discriminative and representative to distinguish from other subcategories? (2) How many discriminative regions are necessary to achieve the best categorization performance? It is still difficult to address these two problems adaptively and intelligently. Artificial prior and experimental validation are widely used in existing mainstream methods to discover which and how many regions to gaze. However, their applications extremely restrict the usability and scalability of the methods. To address the above two problems, this paper proposes a multi-scale and multi-granularity deep reinforcement learning approach (M2DRL), which learns multi-granularity discriminative region attention and multi-scale region-based feature representation. Its main contributions are as follows: (1) Multi-granularity discriminative localization is proposed to localize the distinctions via a two-stage deep reinforcement learning approach, which discovers the discriminative regions with multiple granularities in a hierarchical manner (which problem), and determines the number of discriminative regions in an automatic and adaptive manner (how many problem). (2) Multi-scale representation learning helps to localize regions in different scales as well as encode images in different scales, boosting the fine-grained visual categorization performance. (3) Semantic reward function is proposed to drive M2DRL to fully capture the salient and conceptual visual information, via jointly considering attention and category information in the reward function. It allows the deep reinforcement learning to localize the distinctions in a weakly supervised manner or even an unsupervised manner. (4) Unsupervised discriminative localization is further explored to avoid the heavy labor consumption of annotating, and extremely strengthen the usability and scalability of our M2DRL approach. Compared with state-of-the-art methods on two widely-used fine-grained visual categorization datasets, our M2DRL approach achieves the best categorization accuracy.

Learning a Mixture of Granularity-Specific Experts for Fine-Grained Categorization

Enhancing Mixture-of-Experts by Leveraging Attention for Fine-Grained Recognition

Fine-Grained Visual Categorization With Fine-Tuned Segmentation

Adversarial Mixture Of Experts with Category Hierarchy Soft Constraint

Piecewise Classifier Mappings: Learning Fine-Grained Learners for Novel Categories with Few Examples

Multiple Granularity Descriptors for Fine-Grained Categorization.

Fine-grained Category Discovery under Coarse-grained Supervision with Hierarchical Weighted Self-contrastive Learning

Which Looks Like Which: Exploring Inter-Class Relationships In Fine-Grained Visual Categorization

Learning Regions and Descriptors for Fine-grained Recognition

Learning More Generalized Experts by Merging Experts in Mixture-of-Experts

Which and How Many Regions to Gaze: Focus Discriminative Regions for Fine-Grained Visual Categorization

ECMEE: Expert Constrained Multi-Expert Ensembles with Category Entropy Minimization for long-tailed visual recognition

Part Attentioned Fine-Grained Categorization Using Context Sensitive Flow

Friend or Foe: Fine-Grained Categorization With Weak Supervision.

Low-Rank Mixture-of-Experts for Continual Medical Image Segmentation

Deep Mixture of Diverse Experts for Large-Scale Visual Recognition.

Dynamic collaborative learning with heterogeneous knowledge transfer for long-tailed visual recognition

Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts

Few-shot Learning for Domain-specific Fine-grained Image Classification

Fine-Grained Recognition With Learnable Semantic Data Augmentation

Real Time Fine-Grained Categorization with Accuracy and Interpretability.