Abstract:Deep neural networks (DNNs) involve advanced image processing but depend on large quantities of high-quality labeled data. The presence of noisy data significantly degrades the DNN model performance. In the medical field, where model accuracy is crucial and labels for pathological images are scarce and expensive to obtain, the need to handle noisy data is even more urgent. Deep networks exhibit a memorization effect, they tend to prioritize remembering clean labels initially. Therefore, early stopping is highly effective in managing learning with noisy labels. Previous research has often concentrated on developing robust loss functions or implementing training constraints to mitigate the impact of noisy labels; however, such approaches have frequently resulted in underfitting. We propose using knowledge distillation to slow the learning process of the target network rather than preventing late-stage training from being affected by noisy labels. In this paper, we introduce a data sample self-selection strategy based on early stopping to filter out most of the noisy data. Additionally, we employ the distillation training method with dual teacher networks to ensure the steady learning of the student network. The experimental results show that our method outperforms current state-of-the-art methods for handling noisy labels on both synthetic and real-world noisy datasets. In particular, on the real-world pathological image dataset Chaoyang, the highest classification accuracy increased by 2.39 %. Our method leverages the model's predictions based on training history to select cleaner datasets and retrains them using these cleaner datasets, significantly mitigating the impact of noisy labels on model performance.

No regret sample selection with noisy labels

Mitigating Memorization in Sample Selection for Learning with Noisy Labels

Noise is the Fatal Poison: A Noise-aware Network for Noisy Dataset Classification

Sample self-selection using dual teacher networks for pathological image classification with noisy labels

Label-noise learning via uncertainty-aware neighborhood sample selection

Knockoffs-SPR: Clean Sample Selection in Learning with Noisy Labels

An Ensemble Noise-Robust K-fold Cross-Validation Selection Method for Noisy Labels

Debiased Sample Selection for Combating Noisy Labels

An improved sample selection framework for learning with noisy labels

Late Stopping: Avoiding Confidently Learning from Mislabeled Examples

Noise‐robust neural networks for medical image segmentation by dual‐strategy sample selection

Sample selection with noise rate estimation in noise learning of medical image analysis

A Gradient-based Approach for Online Robust Deep Neural Network Training with Noisy Labels

Combating Label Noise With A General Surrogate Model For Sample Selection

PASS: Peer-Agreement based Sample Selection for training with Noisy Labels

CMW-Net:an Adaptive Robust Algorithm for Sample Selection and Label Correction

USDNL: Uncertainty-Based Single Dropout in Noisy Label Learning.

Carpe Diem, Seize the Samples Uncertain "at the Moment" for Adaptive Batch Selection

Learning from Long-Tailed Noisy Data with Sample Selection and Balanced Loss.

Noisy Early Stopping for Noisy Labels

Small-Vote Sample Selection for Label-Noise Learning