Abstract:Deep neural networks have achieved significant success in the artificial intelligence community and various downstream tasks. They encode images or texts into dense feature representations and are supervised by a large amount of labeled data. Due to the expensiveness of high-quality labeled data, a huge number of easy-to-access instances are collected to conduct supervised learning. However, they have not been annotated by experts and thus can contain numerous noisy instances, which will degrade the performance. To learn robust feature representations despite misleading noisy labels, we employ supervised contrastive learning to directly perform supervision in the hidden space, rather than in the prediction space like the prevalent cross-entropy loss function. However, cutting-edge noisy label learning methods with supervised contrastive learning always discard the data considered to be noisy, and thus cannot tolerate high-ratio noisy datasets. Therefore, we propose a novel training strategy named Supervised Contrastive Learning with Corrected Labels (S cl ^2 ) to defend against the attack of noisy labels. S cl ^2 corrects the noisy labels with an empirical small-loss assumption and conducts supervised contrastive learning using these corrected data. Specifically, we employ the generated soft labels as supervisory information to facilitate our implementation of supervised contrastive learning. This expansion of contrastive learning ensures the integrity of the supervisory information while effectively enhancing the learning process. In addition, samples sharing the same soft labels are treated as positive sample pairs, while those with different soft labels are considered to be negative sample pairs. With this strategy, the representations from neural networks keep the local discrimination in one mini-batch. Besides, we also employ a prototype contrastive learning technique to ensure global discrimination. Our S cl ^2 has demonstrated excellent performance on numerous benchmark datasets, showcasing its effectiveness in various standardized evaluation scenarios. Additionally, our model has proven to be highly valuable when applied to real-world noisy datasets.

Adaptive Contrastive Learning for Learning Robust Representations under Label Noise.

Contrastive Learning Improves Model Robustness Under Label Noise

Selective-Supervised Contrastive Learning with Noisy Labels

Full-Attention Driven Graph Contrastive Learning: with Effective Mutual Information Insight

ECLB: Efficient contrastive learning on bi-level for noisy labels

Supervised contrastive learning with corrected labels for noisy label learning

Label Correction Using Contrastive Prototypical Classifier for Noisy Label Learning.

Noise is Also Useful: Negative Correlation-Steered Latent Contrastive Learning

SCNet: semi-supervised and contrastive learning against noisy labels with two selection strategy approach

Channel-Wise Contrastive Learning for Learning with Noisy Labels

Twin Contrastive Learning with Noisy Labels

Progressively Robust Loss for Deep Learning with Noisy Labels

Representation Learning from Noisy User-Tagged Data for Sentiment Classification

ACE: A Coarse-to-Fine Learning Framework for Reliable Representation Learning Against Label Noise

SV-Learner: Support-Vector Contrastive Learning for Robust Learning with Noisy Labels

Partially View-aligned Representation Learning with Noise-robust Contrastive Loss

Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild.

NaCL: noise-robust cross-domain contrastive learning for unsupervised domain adaptation

Jo-SRC: A Contrastive Approach for Combating Noisy Labels

Learning from Noisy Labels with Complementary Loss Functions

Fine-Grained Classification with Noisy Labels