Abstract:Label noise is ubiquitous in many real-world scenarios which often misleads training algorithm and brings about the degraded classification performance. Therefore, many approaches have been proposed to correct the loss function given corrupted labels to combat such label noise. Among them, a trend of works achieve this goal by unbiasedly estimating the data centroid, which plays an important role in constructing an unbiased risk estimator for minimization. However, they usually handle the noisy labels in different classes all at once, so the local information inherited by each class is ignored which often leads to unsatisfactory performance. To address this defect, this paper presents a novel robust learning algorithm dubbed "Class-Wise Denoising" (CWD), which tackles the noisy labels in a class-wise way to ease the entire noise correction task. Specifically, two virtual auxiliary sets are respectively constructed by presuming that the positive and negative labels in the training set are clean, so the original false-negative labels and false-positive ones are tackled separately. As a result, an improved centroid estimator can be designed which helps to yield more accurate risk estimator. Theoretically, we prove that: 1) the variance in centroid estimation can often be reduced by our CWD when compared with existing methods with unbiased centroid estimator; and 2) the performance of CWD trained on the noisy set will converge to that of the optimal classifier trained on the clean set with a convergence rate O(1/vn) )where n is the number of the training examples. These sound theoretical properties critically enable our CWD to produce the improved classification performance under label noise, which is also demonstrated by the comparisons with ten representative state-of-the-art methods on a variety of benchmark datasets.

Optimized Gradient Clipping for Noisy Label Learning

How to Prevent the Continuous Damage of Noises to Model Training?

A Gradient-based Approach for Online Robust Deep Neural Network Training with Noisy Labels

Mitigating Memorization of Noisy Labels by Clipping the Model Prediction

A Model-Agnostic Approach for Learning with Noisy Labels of Arbitrary Distributions

High Probability Analysis for Non-Convex Stochastic Optimization with Clipping

Learning with noisy labels via logit adjustment based on gradient prior method

Dynamic training for handling textual label noise

Safeguarded Dynamic Label Regression for Generalized Noisy Supervision

Limited Gradient Descent: Learning With Noisy Labels

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions

Gradient Clipping Improves AdaGrad when the Noise Is Heavy-Tailed

Safeguarded Dynamic Label Regression for Noisy Supervision

Gradient optimization for object detection in learning with noisy labels

One-step Noisy Label Mitigation

Adaptive Textual Label Noise Learning Based on Pre-trained Models

Class-Wise Denoising for Robust Learning under Label Noise

Learning with Noisy Labels Via Sparse Regularization

Analyze the Robustness of Classifiers under Label Noise

Dynamic Loss For Robust Learning

Improved Analysis of Clipping Algorithms for Non-convex Optimization