Abstract:Object categories inherently form a hierarchy with different levels of concept abstraction, especially for fine-grained categories. For example, birds (Aves) can be categorized according to a four-level hierarchy of order, family, genus, and species. This hierarchy encodes rich correlations among various categories across different levels, which can effectively regularize the semantic space and thus make prediction less ambiguous. However, previous studies of fine-grained image recognition primarily focus on categories of one certain level and usually overlook this correlation information. In this work, we investigate simultaneously predicting categories of different levels in the hierarchy and integrating this structured correlation information into the deep neural network by developing a novel Hierarchical Semantic Embedding (HSE) framework. Specifically, the HSE framework sequentially predicts the category score vector of each level in the hierarchy, from highest to lowest. At each level, it incorporates the predicted score vector of the higher level as prior knowledge to learn finer-grained feature representation. During training, the predicted score vector of the higher level is also employed to regularize label prediction by using it as soft targets of corresponding sub-categories. To evaluate the proposed framework, we organize the 200 bird species of the Caltech-UCSD birds dataset with the four-level category hierarchy and construct a large-scale butterfly dataset that also covers four level categories. Extensive experiments on these two and the newly-released VegFru datasets demonstrate the superiority of our HSE framework over the baseline methods and existing competitors.

Joint Embeddings of Hierarchical Categories and Entities

Joint Embedding of Hierarchical Categories and Entities for Concept Categorization and Dataless Classification

Bilinear Joint Learning of Word and Entity Embeddings for Entity Linking.

Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding

Knowledge Graph Embedding for Hierarchical Entities Based on Auto-Embedding Size

Hierarchical Embedding Methods for Large-scale Complex Knowledge Graphs

Embedding Hierarchical Tree Structure of Concepts in Knowledge Graph Embedding

Joint embedding in Hierarchical distance and semantic representation learning for link prediction

Embedding Hierarchical Structures for Venue Category Representation

HyperJOIE: Two-View Hyperbolic Knowledge Graph Embedding with Entities and Concepts Jointly

Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding

Learning Visual Hierarchies with Hyperbolic Embeddings

Employing Latent Categories of Entities for Knowledge Graph Embeddings with Contrastive Learning

Jointly Learning Entity and Relation Representations for Entity Alignment

Hierarchical Metadata-Aware Document Categorization under Weak Supervision

EmbeddingTree: Hierarchical Exploration of Entity Features in Embedding

Joint Entity–relation Knowledge Embedding Via Cost-Sensitive Learning

Joint Hierarchical Category Structure Learning and Large-Scale Image Classification

Knowledge Graph Representation with Jointly Structural and Textual Encoding

Joint Embedding of Words and Category Labels for Hierarchical Multi-label Text Classification

Iterative Entity Alignment via Joint Knowledge Embeddings