Abstract:Fine-grained image classification is a challenging task due to the large intra-class variance and small inter-class variance, aiming at recognizing hundreds of sub-categories belonging to the same basic-level category. Most existing fine-grained image classification methods generally learn part detection models to obtain the semantic parts for better classification accuracy. Despite achieving promising results, these methods mainly have two limitations: (1) not all the parts which obtained through the part detection models are beneficial and indispensable for classification, and (2) fine-grained image classification requires more detailed visual descriptions which could not be provided by the part locations or attribute annotations. For addressing the above two limitations, this paper proposes the two-stream model combining vision and language (CVL) for learning latent semantic representations. The vision stream learns deep representations from the original visual information via deep convolutional neural network. The language stream utilizes the natural language descriptions which could point out the discriminative parts or characteristics for each image, and provides a flexible and compact way of encoding the salient visual aspects for distinguishing sub-categories. Since the two streams are complementary, combining the two streams can further achieves better classification accuracy. Comparing with 12 state-of-the-art methods on the widely used CUB-200-2011 dataset for fine-grained image classification, the experimental results demonstrate our CVL approach achieves the best performance.

Combining Latent Semantic Learning and Reduced Hypergraph Learning for Semi-Supervised Image Categorization

Joint Learning of Attended Zero-Shot Features and Visual-Semantic Mapping.

Dual Collaborative Visual-Semantic Mapping for Multi-Label Zero-Shot Image Recognition

Graph-based Multimodal Semi-Supervised Image Classification

Semantic image classification using statistical local spatial relations model

Learning Discriminative Visual Dictionary for Natural Scene Categorization

Supervised latent semantic indexing for document categorization

Semi-supervised Vocabulary-informed Learning

Webly-Supervised Fine-Grained Visual Categorization Via Deep Domain Adaptation.

Semantic Image Segmentation Via Guidance of Image Classification

Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning

Learning Latent Semantic Attributes for Zero-Shot Object Detection.

Exploiting Textual and Visual Features for Image Categorization

Refining local descriptors by embedding semantic information for visual categorization.

Fine-Grained Image Classification Via Combining Vision And Language

Semi-supervised topic modeling for image annotation.

Multimodal visual dictionary learning via heterogeneous latent semantic sparse coding

Weakly Supervised Multimodal Kernel for Categorizing Aerial Photographs

Multiple-instance Learning for Text Categorization Based on Semantic Representation

Deep discriminative image feature learning for cross-modal semantics understanding

Contextual Kernel and Spectral Methods for Learning the Semantics of Images