Abstract:Hierarchical multi-granularity classification (HMC) assigns hierarchical multi-granularity labels to each object and focuses on encoding the label hierarchy, e.g., ["Albatross", "Laysan Albatross"] from coarse-to-fine levels. However, the definition of what is fine-grained is subjective, and the image quality may affect the identification. Thus, samples could be observed at any level of the hierarchy, e.g., ["Albatross"] or ["Albatross", "Laysan Albatross"], and examples discerned at coarse categories are often neglected in the conventional setting of HMC. In this paper, we study the HMC problem in which objects are labeled at any level of the hierarchy. The essential designs of the proposed method are derived from two motivations: (1) learning with objects labeled at various levels should transfer hierarchical knowledge between levels; (2) lower-level classes should inherit attributes related to upper-level superclasses. The proposed combinatorial loss maximizes the marginal probability of the observed ground truth label by aggregating information from related labels defined in the tree hierarchy. If the observed label is at the leaf level, the combinatorial loss further imposes the multi-class cross-entropy loss to increase the weight of fine-grained classification loss. Considering the hierarchical feature interaction, we propose a hierarchical residual network (HRN), in which granularity-specific features from parent levels acting as residual connections are added to features of children levels. Experiments on three commonly used datasets demonstrate the effectiveness of our approach compared to the state-of-the-art HMC approaches and fine-grained visual classification (FGVC) methods exploiting the label hierarchy.

Fine-Grained Entity Typing via Hierarchical Multi Graph Convolutional Networks

Fine-Grained Classification of Named Entities by Fusing Multi-Features.

Type Hierarchy Enhanced Heterogeneous Network Embedding for Fine-Grained Entity Typing in Knowledge Bases

Joint Type Inference on Entities and Relations Via Graph Convolutional Networks

Hierarchical Gate Network for Fine-Grained Visual Recognition.

Multi-view Contrastive Learning for Entity Typing over Knowledge Graphs

Hierarchical Graph Convolutional Networks for Semi-supervised Node Classification

Type-enriched Hierarchical Contrastive Strategy for Fine-Grained Entity Typing

Connecting Embeddings Based on Multiplex Relational Graph Attention Networks for Knowledge Graph Entity Typing

Joint extraction of entities and overlapping relations by improved graph convolutional networks

HfGCN: Hierarchical fused GCN for Joint Entity and Relation Extraction

Neural Architectures for Fine-grained Entity Type Classification

Learning Bi-typed Multi-relational Heterogeneous Graph via Dual Hierarchical Attention Networks

Label Relation Graphs Enhanced Hierarchical Residual Network for Hierarchical Multi-Granularity Classification

Hierarchical Multi-granularity Interaction Graph Convolutional Network for Long Document Classification

Fine-Grained Entity Typing for Relation-Sparsity Entities

Multi-Scale Dynamic Convolutional Network for Knowledge Graph Embedding

Gated Relational Graph Neural Network For Semi-Supervised Learning On Knowledge Graphs

A Multiscale Graph Convolutional Network Using Hierarchical Clustering

Transformer-based Entity Typing in Knowledge Graphs

Entity Relation Interactive Graph Convolutional Network for Knowledge Embedding