Abstract:Robust loss minimization is an important strategy for handling robust learning issue on noisy labels. Current approaches for designing robust losses involve the introduction of noise-robust factors, i.e., hyperparameters, to control the trade-off between noise robustness and learnability. However, finding suitable hyperparameters for different datasets with noisy labels is a challenging and time-consuming task. Moreover, existing robust loss methods usually assume that all training samples share common hyperparameters, which are independent of instances. This limits the ability of these methods to distinguish the individual noise properties of different samples and overlooks the varying contributions of diverse training samples in helping models understand underlying patterns. To address above issues, we propose to assemble robust loss with instance-dependent hyperparameters to improve their noise tolerance with theoretical guarantee. To achieve setting such instance-dependent hyperparameters for robust loss, we propose a meta-learning method which is capable of adaptively learning a hyperparameter prediction function, called noise-aware-robust-loss-adjuster (NARL-Adjuster). Through mutual amelioration between hyperparameter prediction function and classifier parameters in our method, both of them can be simultaneously finely ameliorated and coordinated to attain solutions with good generalization capability. Four SOTA robust loss functions are attempted to be integrated with our algorithm, and comprehensive experiments substantiate the general availability and effectiveness of the proposed method in both its noise tolerance and performance. Meanwhile, the explicit parameterized structure makes the meta-learned prediction function ready to be transferrable and plug-and-play to unseen datasets with noisy labels. Specifically, we transfer our meta-learned NARL-Adjuster to unseen tasks, including several real noisy datasets, and achieve better performance compared with conventional hyperparameter tuning strategy, even with carefully tuned hyperparameters.

Improving Robustness of Random Forest under Label Noise.

FGCM: Noisy Label Learning via Fine-Grained Confidence Modeling

Analyze the Robustness of Classifiers under Label Noise

Robustness and Reliability When Training With Noisy Labels

Robust Long-Tailed Learning under Label Noise

Robust Learning under Hybrid Noise

Improve Noise Tolerance of Robust Loss via Noise-Awareness

Federated Data Quality Assessment Approach: Robust Learning With Mixed Label Noise

PNP: Robust Learning from Noisy Labels by Probabilistic Noise Prediction

Label-Noise Robust Logistic Regression and Its Applications

On the Robustness of Nearest Neighbor with Noisy Data

Label Noise Robust Crowd Counting with Loss Filtering Factor

Making Deep Neural Networks Robust to Label Noise: A Reweighting Loss and Data Filtration

Addressing Long-Tail Noisy Label Learning Problems: a Two-Stage Solution with Label Refurbishment Considering Label Rarity

Exploring Loss Design Techniques For Decision Tree Robustness To Label Noise

Making Deep Neural Networks Robust to Label Noise: Cross-Training with a Novel Loss Function

Adaptive Textual Label Noise Learning Based on Pre-trained Models

BadLabel: A Robust Perspective on Evaluating and Enhancing Label-noise Learning

Progressively Robust Loss for Deep Learning with Noisy Labels

FedNoRo: Towards Noise-Robust Federated Learning by Addressing Class Imbalance and Label Noise Heterogeneity