Abstract:Fine-grained image classification aims to distinguish subcategories belonging to the same basic-level category, such as 200 subcategories belonging to bird. It is a challenging problem in computer vision and multimedia field due to: attribute similarity (e.g. color and texture) among different subcategories and attribute variance (e.g. pose and viewpoint) in the same subcategory. Attribute similarity causes the difficulty to classify different subcategories even for human, while attribute variance causes the learned feature representations chaotic and confused. Naturally, classification can benefit from a hierarchy of subcategories: since going to a coarser granularity leverages high-level semantic features, while going to a finer granularity leverages discriminative and subtle features. Therefore, we propose an attribute hierarchy based multi-task learning (AHMTL) approach, and its main novelties are: (1) Attribute hierarchy: We reassign all images to multi-granularity subcategories automatically, namely coarse-grained, fine-grained and ultra-fine-grained subcategories. Similar fine-grained subcategories are reassigned to the same coarse-grained subcategory according to their attribute similarity, which pays more attention to the high-level semantic feature representations. Simultaneously, the same fine-grained subcategory but with different attributes are divided into different ultra-fine-grained subcategories, which can obtain more discriminative and subtle feature representations for attribute variance. (2) Multi-task learning: A multi-task learning framework is designed to effectively learn robust feature representations by jointly optimizing coarse-grained, fine-grained and ultra-fine-grained image classification tasks. These three level tasks learn coarse to fine feature representations, meaning high-level semantic to subtle features, which can regularize and boost each other to prevent overfitting, and the mutual promotion of them ensures feature representations more discriminative. Compared with more than 10 state-of-the-art methods on two widely-used CUB-200-2011 and Cars-196 datasets, our AHMTL approach achieves the best classification performance.

Discriminative Fast Hierarchical Learning for Multiclass Image Classification

Cost-sensitive Learning of Hierarchical Tree Classifiers for Large-Scale Image Classification and Novel Category Detection

Deep Multi-task Learning for Hierarchical Classification

Joint Hierarchical Category Structure Learning and Large-Scale Image Classification

Constructing Hierarchical Visual Tree for Discriminative Image Representation and Classification.

HD-MTL: Hierarchical Deep Multi-Task Learning for Large-Scale Visual Recognition

Efficient discriminative learning of class hierarchy for many class prediction

Hierarchical Convolutional Neural Network Via Hierarchical Cluster Validity Based Visual Tree Learning

Hierarchical Learning of Tree Classifiers for Large-Scale Plant Species Identification

Hierarchical Gate Network for Fine-Grained Visual Recognition.

Learning Multi-Layer Coarse-to-fine Representations for Large-Scale Image Classification

Attribute Hierarchy Based Multi-Task Learning for Fine-Grained Image Classification

Exploiting Related and Unrelated Tasks for Hierarchical Metric Learning and Image Classification

MMF: Multi-Task Multi-Structure Fusion for Hierarchical Image Classification

Multi-level Discriminative Dictionary Learning with Application to Large Scale Image Classification.

Fast Learning Discriminative Dictionaries for Large-Scale Visual Recognition

Hierarchical Fine-Grained Visual Classification Leveraging Consistent Hierarchical Knowledge

High-Order Distance-Based Multiview Stochastic Learning in Image Classification

Deep Fuzzy Tree for Large-Scale Hierarchical Visual Classification

Attention Tree: Learning Hierarchies of Visual Features for Large-Scale Image Recognition

Integrating hierarchical feature selection and classifier training for multi-label image annotation.