Abstract:Learning delicate feature representation of object parts plays a critical role in fine-grained visual classification tasks. However, advanced deep convolutional neural networks trained for general visual classification tasks usually tend to focus on the coarse-grained information while ignoring the fine-grained one, which is of great significance for learning discriminative representation. In this work, we explore the great merit of multi-modal data in introducing semantic knowledge and sequential analysis techniques in learning hierarchical feature representation for generating discriminative fine-grained features. To this end, we propose a novel approach, termed Channel Cusum Attention ResNet (CCA-ResNet ), for multi-modal joint learning of fine-grained representation. Specifically, we use feature-level multi-modal alignment to connect image and text classification models for joint multi-modal training. Through joint training, image classification models trained with semantic level labels tend to focus on the most discriminative parts, which enhances the cognitive ability of the model. Then, we propose a Channel Cusum Attention (CCA ) mechanism to equip feature maps with hierarchical properties through unsupervised reconstruction of local and global features. The benefits brought by the CCA are in two folds: a) allowing fine-grained features from early layers to be preserved in the forward propagation of deep networks; b) leveraging the hierarchical properties to facilitate multi-modal feature alignment. We conduct extensive experiments to verify that our proposed model can achieve state-of-the-art performance on a series of fine-grained visual classification benchmarks.

Fine-grained Label Learning in Object Detection with Weak Supervision of Captions

Spatial Likelihood Voting with Self-Knowledge Distillation for Weakly Supervised Object Detection.

CISO: Co-iteration Semi-Supervised Learning for Visual Object Detection

Learning Enhanced Features and Inferring Twice for Fine-Grained Image Classification

Fine-Grained Image Captioning with Global-Local Discriminative Objective.

Fine-grained Category Discovery under Coarse-grained Supervision with Hierarchical Weighted Self-contrastive Learning

Leveraging Fine-Grained Labels to Regularize Fine-Grained Visual Classification.

Towards Better Caption Supervision for Object Detection

Weakly Supervised Fine-grained Recognition Based on Combined Learning for Small Data and Coarse Label

Weakly Supervised Learning Of Object-Part Attention Model For Fine-Grained Image Classification

Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization

Weakly Supervised Fine-Grained Categorization With Part-Based Image Representation

Cap2Det: Learning to Amplify Weak Caption Supervision for Object Detection

Weakly Supervised Fine-Grained Image Recognition Based on Multi-Channel Attention and Object Localization

Cyclic-Bootstrap Labeling for Weakly Supervised Object Detection

Learning Hierarchal Channel Attention for Fine-grained Visual Classification.

Learning Two-level Features for Fine-grained Image Classification

Label Hierarchy Constraint Network for Fine-grained Classification

Local-to-Global Semantic Supervised Learning for Image Captioning

Stack-Captioning: Coarse-to-Fine Learning for Image Captioning.

Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification from the Bottom Up