Abstract:We present a detailed study of surrogate losses and algorithms for multi-label learning, supported by $H$-consistency bounds. We first show that, for the simplest form of multi-label loss (the popular Hamming loss), the well-known consistent binary relevance surrogate suffers from a sub-optimal dependency on the number of labels in terms of $H$-consistency bounds, when using smooth losses such as logistic losses. Furthermore, this loss function fails to account for label correlations. To address these drawbacks, we introduce a novel surrogate loss, multi-label logistic loss, that accounts for label correlations and benefits from label-independent $H$-consistency bounds. We then broaden our analysis to cover a more extensive family of multi-label losses, including all common ones and a new extension defined based on linear-fractional functions with respect to the confusion matrix. We also extend our multi-label logistic losses to more comprehensive multi-label comp-sum losses, adapting comp-sum losses from standard classification to the multi-label learning. We prove that this family of surrogate losses benefits from $H$-consistency bounds, and thus Bayes-consistency, across any general multi-label loss. Our work thus proposes a unified surrogate loss framework benefiting from strong consistency guarantees for any multi-label loss, significantly expanding upon previous work which only established Bayes-consistency and for specific loss functions. Additionally, we adapt constrained losses from standard classification to multi-label constrained losses in a similar way, which also benefit from $H$-consistency bounds and thus Bayes-consistency for any multi-label loss. We further describe efficient gradient computation algorithms for minimizing the multi-label logistic loss.

Leveraged Asymmetric Loss with Disambiguation for Multi-label Recognition with One-Positive Annotations

Asymmetric Loss For Multi-Label Classification

Asymmetric Loss for Positive-Unlabeled Learning.

Learning from Multi-Dimensional Partial Labels.

Boosting Single Positive Multi-label Classification with Generalized Robust Loss

Partial Multi-label Learning with Label and Feature Collaboration

Distribution-Balanced Loss for Multi-label Classification in Long-Tailed Datasets

Multi-Label Learning with Stronger Consistency Guarantees

Asymmetric Polynomial Loss For Multi-Label Classification

SPL-MLL: Selecting Predictable Landmarks for Multi-label Learning

A Consistent Lebesgue Measure for Multi-label Learning

Forming Auxiliary High-confident Instance-level Loss to Promote Learning from Label Proportions

Local Positive and Negative Label Correlation Analysis with Label Awareness for Multi-Label Classification

Scalable Label Distribution Learning for Multi-Label Classification

Asymmetric Loss Functions for Learning with Noisy Labels

Penalized Partial Least Square Discriminant Analysis with ℓ 1 - Norm for Multi-Label Data

Enlarged Large Margin Loss for Imbalanced Classification

Large Margin Partial Label Machine

Category-Adaptive Label Discovery and Noise Rejection for Multi-label Recognition with Partial Positive Labels

Pseudo Labels for Imbalanced Multi-Label Learning

Active Negative Loss: A Robust Framework for Learning with Noisy Labels