Abstract:Label noise has been broadly observed in real-world datasets. To mitigate the negative impact of overfitting to label noise for deep models, effective strategies (e.g., re-weighting, or loss rectification) have been broadly applied in prevailing approaches, which have been generally learned under the meta-learning scenario. Despite the robustness of noise achieved by the probabilistic meta-learning models, they usually suffer from model collapse that degenerates generalization performance. In this paper, we propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions as an amortized variational inference problem and derive the evidence lower bound under the meta-learning framework. Specifically, VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable, which can rectify the loss of the noisy sample with the extra randomness regularization and is, therefore, more robust to label noise. To achieve the inference of the rectifying vector, we approximate its conditional posterior with an amortization meta-network. By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function, which can significantly improve the generalization performance. The elaborated meta-network and prior network adhere to the smoothness assumption, enabling the generation of reliable rectification vectors. Given a set of clean meta-data, VRI can be efficiently meta-learned within the bi-level optimization programming. Besides, theoretical analysis guarantees that the meta-network can be efficiently learned with our algorithm. Comprehensive comparison experiments and analyses validate its effectiveness for robust learning with noisy labels, particularly in the presence of open-set noise.

Hierarchical Noise-Tolerant Meta-Learning With Noisy Labels

Learning from Noisy Labels with Decoupled Meta Label Purifier

Noise is the Fatal Poison: A Noise-aware Network for Noisy Dataset Classification

Variational Rectification Inference for Learning with Noisy Labels

Learning with Noisy Labels Via Self-supervised Adversarial Noisy Masking

APAM: Adaptive Pre-training and Adaptive Meta Learning in Language Model for Noisy Labels and Long-tailed Learning

Meta Label Correction for Noisy Label Learning

Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning

Meta Joint Optimization: a Holistic Framework for Noisy-Labeled Visual Recognition

LNL+K: Enhancing Learning with Noisy Labels Through Noise Source Knowledge Integration

Enhanced Meta Label Correction for Coping with Label Corruption

Learning With Noisy Labels Over Imbalanced Subpopulations

Learning to Detect Noisy Labels Using Model-Based Features

Improving deep label noise learning with dual active label correction

Co-LDL: A Co-Training-Based Label Distribution Learning Method for Tackling Label Noise

Learning to Rectify for Robust Learning with Noisy Labels

Learning from Noisy Labels with Noise Modeling Network

Agreement or Disagreement in Noise-tolerant Mutual Learning?

An joint end-to-end framework for learning with noisy labels

Meta Transition Adaptation for Robust Deep Learning with Noisy Labels

DAT: Training Deep Networks Robust to Label-Noise by Matching the Feature Distributions