Abstract:Learning with noisy labels (LNL) has been extensively studied, with existing approaches typically following a framework that alternates between clean sample selection and semi-supervised learning (SSL). However, this approach has a limitation: the clean set selected by the Deep Neural Network (DNN) classifier, trained through self-training, inevitably contains noisy samples. This mixture of clean and noisy samples leads to misguidance in DNN training during SSL, resulting in impaired generalization performance due to confirmation bias caused by error accumulation in sample selection. To address this issue, we propose a method called Collaborative Sample Selection (CSS), which leverages the large-scale pre-trained model CLIP. CSS aims to remove the mixed noisy samples from the identified clean set. We achieve this by training a 2-Dimensional Gaussian Mixture Model (2D-GMM) that combines the probabilities from CLIP with the predictions from the DNN classifier. To further enhance the adaptation of CLIP to LNL, we introduce a co-training mechanism with a contrastive loss in semi-supervised learning. This allows us to jointly train the prompt of CLIP and the DNN classifier, resulting in improved feature representation, boosted classification performance of DNNs, and reciprocal benefits to our Collaborative Sample Selection. By incorporating auxiliary information from CLIP and utilizing prompt fine-tuning, we effectively eliminate noisy samples from the clean set and mitigate confirmation bias during training. Experimental results on multiple benchmark datasets demonstrate the effectiveness of our proposed method in comparison with the state-of-the-art approaches.

Teacher/Student Deep Semi-Supervised Learning for Training with Noisy Labels

Learning with Noisy Labels Via Self-supervised Adversarial Noisy Masking

SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning

Combining Self-Supervised and Supervised Learning with Noisy Labels

Self-training with Noisy Student improves ImageNet classification

Semi-supervised classification of radiology images with NoTeacher: A teacher that is not mean

Learning with noisy labels using collaborative sample selection and contrastive semi-supervised learning

An Overview of Deep Semi-Supervised Learning

STAR: Noisy Semi-Supervised Transfer Learning for Visual Classification

Co-Learning: Towards Semi-Supervised Object Detection with Road-side Cameras

Remote Sensing Image Scene Classification with Noisy Label Distillation

Active Teacher for Semi-Supervised Object Detection

Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels

Semi-supervised 3D Object Detection with Proficient Teachers.

Robust Noisy Label Learning via Two-Stream Sample Distillation

Denoising-Aware Contrastive Learning for Noisy Time Series

Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective

Dual Student: Breaking the Limits of the Teacher in Semi-Supervised Learning

Humble Teachers Teach Better Students for Semi-Supervised Object Detection

Semi-Supervised and Unsupervised Deep Visual Learning: A Survey

Smooth Neighbors on Teacher Graphs for Semi-Supervised Learning