Abstract:Noisy labels are ubiquitous in real-world datasets, especially in the large-scale ones derived from crowdsourcing and web searching. It is challenging to train deep neural networks with noisy datasets since the networks are prone to overfitting the noisy labels during training, resulting in poor generalization performance. During an early learning phase, deep neural networks have been observed to fit the clean samples before memorizing the mislabeled samples. In this paper, we dig deeper into the representation distributions in the early learning phase and find that, regardless of their noisy labels, learned representations of images from the same category still congregate together. Inspired by it, we propose a framework that trains the model with new synthetic samples to mitigate the impact of noisy labels. Specifically, we propose a mixing strategy to create the synthetic samples by aggregating original samples with their top-K nearest neighbours, wherein the weights are calculated using a mixture model learning from the per-sample loss distribution. To enhance the performance in the presence of extreme label noise, we estimate the soft targets by gradually correcting the noisy labels. Furthermore, we demonstrate that the estimated soft targets yield a more accurate approximation to ground truth labels and the proposed method produces a superior quality of learned representations with more separated and clearly bounded clusters. The extensive experiments in two benchmarks (CIFAR-10 and CIFAR-100) and two larg-scale real-world datasets (Clothing1M and Webvision) demonstrate that our approach outperforms the state-of-the-art methods and robustness of the learned representation.

One-step Noisy Label Mitigation

Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective

Learning with Noisy Labels Via Self-supervised Adversarial Noisy Masking

Towards Noise-resistant Object Detection with Noisy Annotations

Learning from Noisy Labels with Decoupled Meta Label Purifier

Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks

Model-agnostic Approaches to Handling Noisy Labels When Training Sound Event Classifiers

Multi-Objective Interpolation Training for Robustness to Label Noise

Mitigating Noisy Supervision Using Synthetic Samples with Soft Labels

An Efficient Noisy Label Learning Method with Semi-supervised Learning: An Efficient Noisy Label Learning Method with Semi-supervised Learning

Dynamic training for handling textual label noise

On Better Detecting and Leveraging Noisy Samples for Learning with Severe Label Noise

A Model-Agnostic Approach for Learning with Noisy Labels of Arbitrary Distributions

Mitigating Memorization in Sample Selection for Learning with Noisy Labels

Optimized Gradient Clipping for Noisy Label Learning

Enhancing Robustness in Learning with Noisy Labels: an Asymmetric Co-Training Approach

Countering Noisy Labels by Learning from Auxiliary Clean Labels

Hybrid Noise-Oriented Multilabel Learning.

Which Strategies Matter for Noisy Label Classification? Insight into Loss and Uncertainty

BadLabel: A Robust Perspective on Evaluating and Enhancing Label-noise Learning

Coordinated Sparse Recovery of Label Noise