Abstract:Training deep neural networks (DNN) with noisy labels is challenging since DNN can easily memorize inaccurate labels, leading to poor generalization ability. Recently, the meta-learning based label correction strategy is widely adopted to tackle this problem via identifying and correcting potential noisy labels with the help of a small set of clean validation data. Although training with purified labels can effectively improve performance, solving the meta-learning problem inevitably involves a nested loop of bi-level optimization between model weights and hyper-parameters (i.e., label distribution). As compromise, previous methods resort to a coupled learning process with alternating update. In this paper, we empirically find such simultaneous optimization over both model weights and label distribution can not achieve an optimal routine, consequently limiting the representation ability of backbone and accuracy of corrected labels. From this observation, a novel multi-stage label purifier named DMLP is proposed. DMLP decouples the label correction process into label-free representation learning and a simple meta label purifier, In this way, DMLP can focus on extracting discriminative feature and label correction in two distinctive stages. DMLP is a plug-and-play label purifier, the purified labels can be directly reused in naive end-to-end network retraining or other robust learning methods, where state-of-the-art results are obtained on several synthetic and real-world noisy datasets, especially under high noise levels. Code is available at https://github.com/yuanpengtu/DMLP.

Understanding and Improving Early Stopping for Learning with Noisy Labels

Noisy Early Stopping for Noisy Labels

Robust early-learning: Hindering the memorization of noisy labels

Tackling Noisy Labels with Network Parameter Additive Decomposition

Learning from Noisy Labels with Decoupled Meta Label Purifier

PADDLES: Phase-Amplitude Spectrum Disentangled Early Stopping for Learning with Noisy Labels

Subclass-Dominant Label Noise: A Counterexample for the Success of Early Stopping

Learning from Noisy Labels Via Dynamic Loss Thresholding

Early Stopping Against Label Noise Without Validation Data

Early-Learning Regularization Prevents Memorization of Noisy Labels

Late Stopping: Avoiding Confidently Learning from Mislabeled Examples

Learning with Noisy Labels Via Self-supervised Adversarial Noisy Masking

On Emergence of Clean-Priority Learning in Early Stopped Neural Networks

Probabilistic End-To-End Noise Correction for Learning with Noisy Labels

Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks

PENCIL: Deep Learning with Noisy Labels

On Better Detecting and Leveraging Noisy Samples for Learning with Severe Label Noise

Dynamic training for handling textual label noise

Limited Gradient Descent: Learning With Noisy Labels

Towards harnessing feature embedding for robust learning with noisy labels

An joint end-to-end framework for learning with noisy labels