Percutaneous treatment of atrial septal defects, muscular ventricular septal defects and patent ductus arteriosus in infants under one year of age.

F. Prada,C. Mortera,J. Bartrons,M. Rissech,L. Jiménez,J. Carretero,J. Llevadias,Mireya Araica

DOI: https://doi.org/10.1016/S1885-5857(09)73272-4

IF: 7.05

2009-09-01

Revista Española de Cardiología

Abstract:

What problem does this paper attempt to address?

Uncertainty-aware Complementary Label Queries for Active Learning

Shengyuan Liu,Ke Chen,Tianlei Hu,Yunqing Mao

DOI: https://doi.org/10.1631/fitee.2200589

2023-01-01

Abstract:Many active learning methods are based on the assumption that a learner simply asks for the true labels of some training data from annotators. Unfortunately, it is expensive to exactly annotate instances in real-world classification tasks. To mitigate this problem, we propose a novel active learning setting, named active learning with complementary labels (ALCL). The ALCL learners only ask yes/no questions in some classes. After receiving answers from annotators, ALCL learners get a few supervised data and more training instances with complementary labels, which only specify one of the classes to which the pattern does not belong. There are two challenging issues in ALCL: the first is how to learn from these complementary labels, and another is how to sample instances to be queried. For the first issue, we propose a novel method that redistributes the weights of instances based on the balance of category contribution to learn from ordinary labels and complementary labels. For the second issue, we propose a weighting mechanism to improve existing uncertainty-based sampling strategies under this novel setup. Experiments on various datasets demonstrate the feasibility of ALCL and the superiority of our approaches.
Complementary label learning based on knowledge distillation

Peng Ying,Zhongnian Li,Renke Sun,Xinzheng Xu

DOI: https://doi.org/10.3934/mbe.2023796

2023-09-19

Abstract:Complementary label learning (CLL) is a type of weakly supervised learning method that utilizes the category of samples that do not belong to a certain class to learn their true category. However, current CLL methods mainly rely on rewriting classification losses without fully leveraging the supervisory information in complementary labels. Therefore, enhancing the supervised information in complementary labels is a promising approach to improve the performance of CLL. In this paper, we propose a novel framework called Complementary Label Enhancement based on Knowledge Distillation (KDCL) to address the lack of attention given to complementary labels. KDCL consists of two deep neural networks: a teacher model and a student model. The teacher model focuses on softening complementary labels to enrich the supervision information in them, while the student model learns from the complementary labels that have been softened by the teacher model. Both the teacher and student models are trained on the dataset that contains only complementary labels. To evaluate the effectiveness of KDCL, we conducted experiments on four datasets, namely MNIST, F-MNIST, K-MNIST and CIFAR-10, using two sets of teacher-student models (Lenet-5+MLP and DenseNet-121+ResNet-18) and three CLL algorithms (PC, FWD and SCL-NL). Our experimental results demonstrate that models optimized by KDCL outperform those trained only with complementary labels in terms of accuracy.
Enhancing Label Sharing Efficiency in Complementary-Label Learning with Label Augmentation

Wei-I Lin,Gang Niu,Hsuan-Tien Lin,Masashi Sugiyama

DOI: https://doi.org/10.48550/arXiv.2305.08344

2023-05-15

Abstract:Complementary-label Learning (CLL) is a form of weakly supervised learning that trains an ordinary classifier using only complementary labels, which are the classes that certain instances do not belong to. While existing CLL studies typically use novel loss functions or training techniques to solve this problem, few studies focus on how complementary labels collectively provide information to train the ordinary classifier. In this paper, we fill the gap by analyzing the implicit sharing of complementary labels on nearby instances during training. Our analysis reveals that the efficiency of implicit label sharing is closely related to the performance of existing CLL models. Based on this analysis, we propose a novel technique that enhances the sharing efficiency via complementary-label augmentation, which explicitly propagates additional complementary labels to each instance. We carefully design the augmentation process to enrich the data with new and accurate complementary labels, which provide CLL models with fresh and valuable information to enhance the sharing efficiency. We then verify our proposed technique by conducting thorough experiments on both synthetic and real-world datasets. Our results confirm that complementary-label augmentation can systematically improve empirical performance over state-of-the-art CLL models.

Machine Learning
Complementary to Multiple Labels: A Correlation-Aware Correction Approach

Yi Gao,Miao Xu,Min-Ling Zhang

DOI: https://doi.org/10.1109/TPAMI.2024.3416384

2023-02-25

Abstract:\textit{Complementary label learning} (CLL) requires annotators to give \emph{irrelevant} labels instead of relevant labels for instances. Currently, CLL has shown its promising performance on multi-class data by estimating a transition matrix. However, current multi-class CLL techniques cannot work well on multi-labeled data since they assume each instance is associated with one label while each multi-labeled instance is relevant to multiple labels. Here, we show theoretically how the estimated transition matrix in multi-class CLL could be distorted in multi-labeled cases as they ignore co-existing relevant labels. Moreover, theoretical findings reveal that calculating a transition matrix from label correlations in \textit{multi-labeled CLL} (ML-CLL) needs multi-labeled data, while this is unavailable for ML-CLL. To solve this issue, we propose a two-step method to estimate the transition matrix from candidate labels. Specifically, we first estimate an initial transition matrix by decomposing the multi-label problem into a series of binary classification problems, then the initial transition matrix is corrected by label correlations to enforce the addition of relationships among labels. We further show that the proposal is classifier-consistent, and additionally introduce an MSE-based regularizer to alleviate the tendency of BCE loss overfitting to noises. Experimental results have demonstrated the effectiveness of the proposed method.

Machine Learning
Complementary Labels Learning with Augmented Classes

Zhongnian Li,Mengting Xu,Xinzheng Xu,Daoqiang Zhang

DOI: https://doi.org/10.1016/j.knosys.2024.111570

IF: 8.139

2024-02-25

Knowledge-Based Systems

Abstract:Complementary Labels Learning (CLL) arises in many real-world tasks such as private questions classification and online learning, which aims to alleviate the annotation cost compared with standard supervised learning. Unfortunately, most previous CLL algorithms were in a stable environment rather than an open and dynamic scenarios, where data collected from unseen augmented classes in the training process might emerge in the testing phase. In this paper, we propose a novel problem setting called Complementary Labels Learning with Augmented Classes (CLLAC), which brings the challenge that classifiers trained by complementary labels should not only be able to classify the instances from observed classes accurately, but also recognize the instance from the Augmented Classes in the testing phase. Specifically, by using unlabeled data, we propose an unbiased estimator of classification risk for CLLAC, which is guaranteed to be provably consistent. Moreover, we provide generalization error bound for proposed method which shows that the optimal parametric convergence rate is achieved for estimation error. Finally, the experimental results on several benchmark datasets verify the effectiveness of the proposed method.

computer science, artificial intelligence
CLImage: Human-Annotated Datasets for Complementary-Label Learning

Hsiu-Hsuan Wang,Tan-Ha Mai,Nai-Xuan Ye,Wei-I Lin,Hsuan-Tien Lin

2024-06-22

Abstract:Complementary-label learning (CLL) is a weakly-supervised learning paradigm that aims to train a multi-class classifier using only complementary labels, which indicate classes to which an instance does not belong. Despite numerous algorithmic proposals for CLL, their practical applicability remains unverified for two reasons. Firstly, these algorithms often rely on assumptions about the generation of complementary labels, and it is not clear how far the assumptions are from reality. Secondly, their evaluation has been limited to synthetic datasets. To gain insights into the real-world performance of CLL algorithms, we developed a protocol to collect complementary labels from human annotators. Our efforts resulted in the creation of four datasets: CLCIFAR10, CLCIFAR20, CLMicroImageNet10, and CLMicroImageNet20, derived from well-known classification datasets CIFAR10, CIFAR100, and TinyImageNet200. These datasets represent the very first real-world CLL datasets. Through extensive benchmark experiments, we discovered a notable decrease in performance when transitioning from synthetic datasets to real-world datasets. We investigated the key factors contributing to the decrease with a thorough dataset-level ablation study. Our analyses highlight annotation noise as the most influential factor in the real-world datasets. In addition, we discover that the biased-nature of human-annotated complementary labels and the difficulty to validate with only complementary labels are two outstanding barriers to practical CLL. These findings suggest that the community focus more research efforts on developing CLL algorithms and validation schemes that are robust to noisy and biased complementary-label distributions.

Machine Learning,Computer Vision and Pattern Recognition
Complementary Classifier Induced Partial Label Learning

Yuheng Jia,Chongjie Si,Min-ling Zhang

DOI: https://doi.org/10.1145/3580305.3599282

2023-05-17

Abstract:In partial label learning (PLL), each training sample is associated with a set of candidate labels, among which only one is valid. The core of PLL is to disambiguate the candidate labels to get the ground-truth one. In disambiguation, the existing works usually do not fully investigate the effectiveness of the non-candidate label set (a.k.a. complementary labels), which accurately indicates a set of labels that do not belong to a sample. In this paper, we use the non-candidate labels to induce a complementary classifier, which naturally forms an adversarial relationship against the traditional PLL classifier, to eliminate the false-positive labels in the candidate label set. Besides, we assume the feature space and the label space share the same local topological structure captured by a dynamic graph, and use it to assist disambiguation. Extensive experimental results validate the superiority of the proposed approach against state-of-the-art PLL methods on 4 controlled UCI data sets and 6 real-world data sets, and reveal the usefulness of complementary learning in PLL. The code has been released in the link <a class="link-external link-https" href="https://github.com/Chongjie-Si/PL-CL" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Computer Vision and Pattern Recognition
Self-adaptive label discovery and multi-view fusion for complementary label learning

Long Tang,Pengfei Yan,Yingjie Tian,Pano M Pardalos

DOI: https://doi.org/10.1016/j.neunet.2024.106763

2024-10-02

Abstract:Unlike traditional supervised classification, complementary label learning (CLL) operates under a weak supervision framework, where each sample is annotated by excluding several incorrect labels, known as complementary labels (CLs). Despite reducing the labeling burden, CLL always suffers a decline in performance due to the weakened supervised information. To overcome such limitations, in this study, a multi-view fusion and self-adaptive label discovery based CLL method (MVSLDCLL) is proposed. The self-adaptive label discovery strategy leverages graph-based semi-supervised learning to capture the label distribution of each training sample as a convex combination of all its potential labels. The multi-view fusion module is designed to adapt to various views of feature representations. In specific, it minimizes the discrepancies of label projections between pairwise views, aligning with the consensus principle. Additionally, a straightforward mechanism inspired by a teamwork analogy is proposed to incorporate view-discrepancy for each sample. Experimental results demonstrate that MVSLDCLL learns more discriminative label distribution and achieves significantly higher accuracies compared to state-of-the-art CLL methods. Ablation study has also been performed to validate the effectiveness of both the self-adaptive label discovery strategy and the multi-view fusion module.
Class-Imbalanced Complementary-Label Learning via Weighted Loss

Meng Wei,Yong Zhou,Zhongnian Li,Xinzheng Xu

DOI: https://doi.org/10.1016/j.neunet.2023.07.030

2023-06-17

Abstract:Complementary-label learning (CLL) is widely used in weakly supervised classification, but it faces a significant challenge in real-world datasets when confronted with class-imbalanced training samples. In such scenarios, the number of samples in one class is considerably lower than in other classes, which consequently leads to a decline in the accuracy of predictions. Unfortunately, existing CLL approaches have not investigate this problem. To alleviate this challenge, we propose a novel problem setting that enables learning from class-imbalanced complementary labels for multi-class classification. To tackle this problem, we propose a novel CLL approach called Weighted Complementary-Label Learning (WCLL). The proposed method models a weighted empirical risk minimization loss by utilizing the class-imbalanced complementary labels, which is also applicable to multi-class imbalanced training samples. Furthermore, we derive an estimation error bound to provide theoretical assurance. To evaluate our approach, we conduct extensive experiments on several widely-used benchmark datasets and a real-world dataset, and compare our method with existing state-of-the-art methods. The proposed approach shows significant improvement in these datasets, even in the case of multiple class-imbalanced scenarios. Notably, the proposed method not only utilizes complementary labels to train a classifier but also solves the problem of class imbalance.

Machine Learning
GraphCoCo: Graph Complementary Contrastive Learning

Yue Ding,Chentao Wu,Xinyu Lu,Jiawei Sun,Xiang Yu,Jie Li,Ruoxin Chen,Junchi Yan

DOI: https://doi.org/10.48550/arXiv.2203.12821

Abstract:Graph Contrastive Learning (GCL) has shown promising performance in graph representation learning (GRL) without the supervision of manual annotations. GCL can generate graph-level embeddings by maximizing the Mutual Information (MI) between different augmented views of the same graph (positive pairs). However, we identify a obstacle that the optimization of InfoNCE loss only concentrates on a few embeddings dimensions, limiting the distinguishability of embeddings in downstream graph classiﬁcation tasks. This paper proposes an effective graph co mplementary co ntrastive learning approach named GraphCoCo to tackle the above issue. Speciﬁcally, we set the embedding of the ﬁrst augmented view as the anchor embedding to localize “highlighted” dimensions (i.e., the dimensions contribute most in similarity measurement). Then remove these dimensions in the embeddings of the second augmented view to discover neglected complementary representations. Therefore, the combination of anchor and complementary embeddings signiﬁcantly improves the performance in downstream tasks. Comprehensive experiments on various benchmark datasets are conducted to demonstrate the effectiveness of GraphCoCo, and the results show that our model outperforms the state-of-the-art methods.

Computer Science
Generative-Discriminative Complementary Learning

Yanwu Xu,Mingming Gong,Junxiang Chen,Tongliang Liu,Kun Zhang,Kayhan Batmanghelich

DOI: https://doi.org/10.48550/arXiv.1904.01612

2019-09-12

Abstract:Majority of state-of-the-art deep learning methods are discriminative approaches, which model the conditional distribution of labels given inputs features. The success of such approaches heavily depends on high-quality labeled instances, which are not easy to obtain, especially as the number of candidate classes increases. In this paper, we study the complementary learning problem. Unlike ordinary labels, complementary labels are easy to obtain because an annotator only needs to provide a yes/no answer to a randomly chosen candidate class for each instance. We propose a generative-discriminative complementary learning method that estimates the ordinary labels by modeling both the conditional (discriminative) and instance (generative) distributions. Our method, we call Complementary Conditional GAN (CCGAN), improves the accuracy of predicting ordinary labels and can generate high-quality instances in spite of weak supervision. In addition to the extensive empirical studies, we also theoretically show that our model can retrieve the true conditional distribution from the complementarily-labeled data.

Machine Learning
Label Anchored Contrastive Learning for Language Understanding

Zhenyu Zhang,Yuming Zhao,Meng Chen,Xiaodong He

DOI: https://doi.org/10.48550/arXiv.2205.10227

2022-04-26

Abstract:Contrastive learning (CL) has achieved astonishing progress in computer vision, speech, and natural language processing fields recently with self-supervised learning. However, CL approach to the supervised setting is not fully explored, especially for the natural language understanding classification task. Intuitively, the class label itself has the intrinsic ability to perform hard positive/negative mining, which is crucial for CL. Motivated by this, we propose a novel label anchored contrastive learning approach (denoted as LaCon) for language understanding. Specifically, three contrastive objectives are devised, including a multi-head instance-centered contrastive loss (ICL), a label-centered contrastive loss (LCL), and a label embedding regularizer (LER). Our approach does not require any specialized network architecture or any extra data augmentation, thus it can be easily plugged into existing powerful pre-trained language models. Compared to the state-of-the-art baselines, LaCon obtains up to 4.1% improvement on the popular datasets of GLUE and CLUE benchmarks. Besides, LaCon also demonstrates significant advantages under the few-shot and data imbalance settings, which obtains up to 9.4% improvement on the FewGLUE and FewCLUE benchmarking tasks.

Computation and Language
libcll: an Extendable Python Toolkit for Complementary-Label Learning

Nai-Xuan Ye,Tan-Ha Mai,Hsiu-Hsuan Wang,Wei-I Lin,Hsuan-Tien Lin

2024-11-19

Abstract:Complementary-label learning (CLL) is a weakly supervised learning paradigm for multiclass classification, where only complementary labels -- indicating classes an instance does not belong to -- are provided to the learning algorithm. Despite CLL's increasing popularity, previous studies highlight two main challenges: (1) inconsistent results arising from varied assumptions on complementary label generation, and (2) high barriers to entry due to the lack of a standardized evaluation platform across datasets and algorithms. To address these challenges, we introduce \texttt{libcll}, an extensible Python toolkit for CLL research. \texttt{libcll} provides a universal interface that supports a wide range of generation assumptions, both synthetic and real-world datasets, and key CLL algorithms. The toolkit is designed to mitigate inconsistencies and streamline the research process, with easy installation, comprehensive usage guides, and quickstart tutorials that facilitate efficient adoption and implementation of CLL techniques. Extensive ablation studies conducted with \texttt{libcll} demonstrate its utility in generating valuable insights to advance future CLL research.

Machine Learning,Artificial Intelligence,Computer Vision and Pattern Recognition
CoMAL: Contrastive Active Learning for Multi-Label Text Classification

Cheng Peng,Haobo Wang,Ke Chen,Lidan Shou,Chang Yao,Runze Wu,Gang Chen

DOI: https://doi.org/10.1145/3637528.3671754

2024-01-01

Abstract:Multi-label text classification (MLTC) allows a given text to be associated with multiple labels, which well suits many real-world data mining scenarios. However, the annotation effort of MLTC is inevitably expensive and time-consuming. Although multi-label active learning provides a cost-effective solution, it still faces two major challenges: (i) constructing decent feature space to distinguish the confusing semantics of different labels; (ii) defining proper sampling criteria to measure a sample's joint effect over the entire label space. To bridge these gaps, we propose a Contrastive Multi-label Active Learning framework (CoMAL) that gives an effective data acquisition strategy. Specifically, a contrastive decoupling mechanism is introduced to fully release the semantic information of multiple labels into the latent space. Then, we devise a hybrid criterion that balances two data value measures: (i) similarity-enhanced label cardinality inconsistency reflects the uncertainty of data predictions. (ii) positive feature diversity evaluates the positive-propensity semantic diversity to handle the label sparsity. Extensive experiments demonstrate that our CoMAL outperforms the current state-of-the-art multi-label active learning approaches. Code for CoMAL is available at https://github.com/chengzju/CoMAL.
CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning

Xin Zhang,Zixuan Liu,Kaiwen Xiao,Tian Shen,Junzhou Huang,Wei Yang,Dimitris Samaras,Xiao Han

DOI: https://doi.org/10.48550/arXiv.2111.11652

IF: 5.414

2021-11-23

Machine Learning

Abstract:Labels are costly and sometimes unreliable. Noisy label learning, semi-supervised learning, and contrastive learning are three different strategies for designing learning processes requiring less annotation cost. Semi-supervised learning and contrastive learning have been recently demonstrated to improve learning strategies that address datasets with noisy labels. Still, the inner connections between these fields as well as the potential to combine their strengths together have only started to emerge. In this paper, we explore further ways and advantages to fuse them. Specifically, we propose CSSL, a unified Contrastive Semi-Supervised Learning algorithm, and CoDiM (Contrastive DivideMix), a novel algorithm for learning with noisy labels. CSSL leverages the power of classical semi-supervised learning and contrastive learning technologies and is further adapted to CoDiM, which learns robustly from multiple types and levels of label noise. We show that CoDiM brings consistent improvements and achieves state-of-the-art results on multiple benchmarks.
Learning From a Complementary-Label Source Domain: Theory and Algorithms

Yiyang Zhang,Feng Liu,Zhen Fang,Bo Yuan,Guangquan Zhang,Jie Lu

DOI: https://doi.org/10.1109/tnnls.2021.3086093

IF: 14.255

2021-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:In unsupervised domain adaptation (UDA), a classifier for the target domain is trained with massive true-label data from the source domain and unlabeled data from the target domain. However, collecting true-label data in the source domain can be expensive and sometimes impractical. Compared to the true label (TL), a complementary label (CL) specifies a class that a pattern does not belong to, and hence, collecting CLs would be less laborious than collecting TLs. In this article, we propose a novel setting where the source domain is composed of complementary-label data, and a theoretical bound of this setting is provided. We consider two cases of this setting: one is that the source domain only contains complementary-label data [completely complementary UDA (CC-UDA)] and the other is that the source domain has plenty of complementary-label data and a small amount of true-label data [partly complementary UDA (PC-UDA)]. To this end, a complementary label adversarial network (CLARINET) is proposed to solve CC-UDA and PC-UDA problems. CLARINET maintains two deep networks simultaneously, with one focusing on classifying the complementary-label source data and the other taking care of the source-to-target distributional adaptation. Experiments show that CLARINET significantly outperforms a series of competent baselines on handwritten digit-recognition and object-recognition tasks.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
ECLB: Efficient contrastive learning on bi-level for noisy labels

Juwei Guan,Jiaxiang Liu,Shuying Huang,Yong Yang

DOI: https://doi.org/10.1016/j.knosys.2024.112128

IF: 8.139

2024-06-17

Knowledge-Based Systems

Abstract:For contrastive learning has achieved remarkable success in self-supervised and supervised informative representation, learning with noisy labels based on contrastive learning is becoming the research consensus. However, under noisy labels, how to efficiently leverage informative representation of various levels and how to effectively screen reliable positive pairs for the optimization of the contrastive learning model are still challenges. To address these issues, we innovatively propose a method named efficient contrastive learning on bi-level for noisy labels (ECLB), which is jointly implemented by both self-supervised and supervised contrastive learning. For the accessible informative representation, we propose to perform contrastive learning at two different levels: (1) feature level, where feature representation is jointly optimized by supervised and self-supervised feature contrastive loss; (2) label level, where feature and label representation are optimized by label distribution supervised contrastive loss. Furthermore, to alleviate the impact of noisy labels on the selection of reliable positive pairs in supervised contrastive learning and to reduce labor cost and computational complexity, we propose an efficient adaptive mask, which is dynamically generated by label self-equality mask, prediction self-equality mask, label-prediction equality mask, and feature similarity mask. Extensive experiments show that our proposed method outperforms other state-of-the-art methods in terms of robustness and generalization. Our code is publicly available at: https://github.com/whyandbecause/ECLB

computer science, artificial intelligence
Learning with Biased Complementary Labels

Xiyu Yu,Tongliang Liu,Mingming Gong,Dacheng Tao

DOI: https://doi.org/10.48550/arXiv.1711.09535

2018-08-08

Abstract:In this paper, we study the classification problem in which we have access to easily obtainable surrogate for true labels, namely complementary labels, which specify classes that observations do \textbf{not} belong to. Let $Y$ and $\bar{Y}$ be the true and complementary labels, respectively. We first model the annotation of complementary labels via transition probabilities $P(\bar{Y}=i|Y=j), i\neq j\in\{1,\cdots,c\}$, where $c$ is the number of classes. Previous methods implicitly assume that $P(\bar{Y}=i|Y=j), \forall i\neq j$, are identical, which is not true in practice because humans are biased toward their own experience. For example, as shown in Figure 1, if an annotator is more familiar with monkeys than prairie dogs when providing complementary labels for meerkats, she is more likely to employ "monkey" as a complementary label. We therefore reason that the transition probabilities will be different. In this paper, we propose a framework that contributes three main innovations to learning with \textbf{biased} complementary labels: (1) It estimates transition probabilities with no bias. (2) It provides a general method to modify traditional loss functions and extends standard deep neural network classifiers to learn with biased complementary labels. (3) It theoretically ensures that the classifier learned with complementary labels converges to the optimal one learned with true labels. Comprehensive experiments on several benchmark datasets validate the superiority of our method to current state-of-the-art methods.

Machine Learning
Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical

Wei Wang,Takashi Ishida,Yu-Jie Zhang,Gang Niu,Masashi Sugiyama

2024-10-11

Abstract:Complementary-label learning is a weakly supervised learning problem in which each training example is associated with one or multiple complementary labels indicating the classes to which it does not belong. Existing consistent approaches have relied on the uniform distribution assumption to model the generation of complementary labels, or on an ordinary-label training set to estimate the transition matrix in non-uniform cases. However, either condition may not be satisfied in real-world scenarios. In this paper, we propose a novel consistent approach that does not rely on these conditions. Inspired by the positive-unlabeled (PU) learning literature, we propose an unbiased risk estimator based on the Selected-Completely-at-Random assumption for complementary-label learning. We then introduce a risk-correction approach to address overfitting problems. Furthermore, we find that complementary-label learning can be expressed as a set of negative-unlabeled binary classification problems when using the one-versus-rest strategy. Extensive experimental results on both synthetic and real-world benchmark datasets validate the superiority of our proposed approach over state-of-the-art methods.

Machine Learning
Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition

Zi-Hao Zhou,Siyuan Fang,Zi-Jing Zhou,Tong Wei,Yuanyu Wan,Min-Ling Zhang

2024-10-08

Abstract:Long-tailed semi-supervised learning poses a significant challenge in training models with limited labeled data exhibiting a long-tailed label distribution. Current state-of-the-art LTSSL approaches heavily rely on high-quality pseudo-labels for large-scale unlabeled data. However, these methods often neglect the impact of representations learned by the neural network and struggle with real-world unlabeled data, which typically follows a different distribution than labeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. Our framework derives the class-balanced contrastive loss through Gaussian kernel density estimation. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels. By progressively estimating the underlying label distribution and optimizing its alignment with model predictions, we tackle the diverse distribution of unlabeled data in real-world scenarios. Extensive experiments across multiple datasets with varying unlabeled data distributions demonstrate that CCL consistently outperforms prior state-of-the-art methods, achieving over 4% improvement on the ImageNet-127 dataset. Our source code is available at <a class="link-external link-https" href="https://github.com/zhouzihao11/CCL" rel="external noopener nofollow">this https URL</a>

Machine Learning

Percutaneous treatment of atrial septal defects, muscular ventricular septal defects and patent ductus arteriosus in infants under one year of age.

Uncertainty-aware Complementary Label Queries for Active Learning

Complementary label learning based on knowledge distillation

Enhancing Label Sharing Efficiency in Complementary-Label Learning with Label Augmentation

Complementary to Multiple Labels: A Correlation-Aware Correction Approach

Complementary Labels Learning with Augmented Classes

CLImage: Human-Annotated Datasets for Complementary-Label Learning

Complementary Classifier Induced Partial Label Learning

Self-adaptive label discovery and multi-view fusion for complementary label learning

Class-Imbalanced Complementary-Label Learning via Weighted Loss

GraphCoCo: Graph Complementary Contrastive Learning

Generative-Discriminative Complementary Learning

Label Anchored Contrastive Learning for Language Understanding

libcll: an Extendable Python Toolkit for Complementary-Label Learning

CoMAL: Contrastive Active Learning for Multi-Label Text Classification

CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning

Learning From a Complementary-Label Source Domain: Theory and Algorithms

ECLB: Efficient contrastive learning on bi-level for noisy labels

Learning with Biased Complementary Labels

Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical

Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition