Taxes Are All You Need: Integration of Taxonomical Hierarchy Relationships into the Contrastive Loss

Kiran Kokilepersaud,Yavuz Yarici,Mohit Prabhushankar,Ghassan AlRegib

2024-06-11

Abstract:In this work, we propose a novel supervised contrastive loss that enables the integration of taxonomic hierarchy information during the representation learning process. A supervised contrastive loss operates by enforcing that images with the same class label (positive samples) project closer to each other than images with differing class labels (negative samples). The advantage of this approach is that it directly penalizes the structure of the representation space itself. This enables greater flexibility with respect to encoding semantic concepts. However, the standard supervised contrastive loss only enforces semantic structure based on the downstream task (i.e. the class label). In reality, the class label is only one level of a \emph{hierarchy of different semantic relationships known as a taxonomy}. For example, the class label is oftentimes the species of an animal, but between different classes there are higher order relationships such as all animals with wings being ``birds". We show that by explicitly accounting for these relationships with a weighting penalty in the contrastive loss we can out-perform the supervised contrastive loss. Additionally, we demonstrate the adaptability of the notion of a taxonomy by integrating our loss into medical and noise-based settings that show performance improvements by as much as 7%.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The paper introduces a new method of supervised contrastive loss called TaxCL, which incorporates Taxonomical Hierarchy Relationships into the loss function. Traditional supervised contrastive loss only distinguishes positive and negative samples based on the category labels of downstream tasks, while TaxCL considers higher-level semantic relationships beyond the category labels, such as the fact that all winged animals are "birds". By introducing weight penalty into the contrastive loss, the paper demonstrates that this approach can outperform standard supervised contrastive loss and achieve performance improvements of up to 7% in medical and noise-based settings. In addition, the authors analyze the characteristics of the representation space and find that considering the taxonomic structure helps reduce dimension collapse and improve the discrimination between different samples. The paper also demonstrates the generalization ability of TaxCL on various datasets and shows that it can be linearly combined with supervised contrastive loss to further improve performance.