Abstract:Deep supervised learning has achieved remarkable success across a wide range of tasks, yet it remains susceptible to overfitting when confronted with noisy labels. To address this issue, noise-robust loss functions offer an effective solution for enhancing learning in the presence of label noise. In this work, we systematically investigate the limitation of the recently proposed Active Passive Loss (APL), which employs Mean Absolute Error (MAE) as its passive loss function. Despite the robustness brought by MAE, one of its key drawbacks is that it pays equal attention to clean and noisy samples; this feature slows down convergence and potentially makes training difficult, particularly in large-scale datasets. To overcome these challenges, we introduce a novel loss function class, termed Normalized Negative Loss Functions (NNLFs), which serve as passive loss functions within the APL framework. NNLFs effectively address the limitations of MAE by concentrating more on memorized clean samples. By replacing MAE in APL with our proposed NNLFs, we enhance APL and present a new framework called Active Negative Loss (ANL). Moreover, in non-symmetric noise scenarios, we propose an entropy-based regularization technique to mitigate the vulnerability to the label imbalance. Extensive experiments demonstrate that the new loss functions adopted by our ANL framework can achieve better or comparable performance to state-of-the-art methods across various label noise types and in image segmentation tasks. The source code is available at: <a class="link-external link-https" href="https://github.com/Virusdoll/Active-Negative-Loss" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is the problem of over - fitting in deep supervised learning when facing data with noisy labels. Specifically, when there are annotation errors in the data set, deep neural networks (DNNs) may over - fit these wrong labels, resulting in a decline in model performance. To solve this problem, researchers have proposed a variety of noise - resistant loss functions to enhance the model's learning ability under noisy labels. ### Main contributions of the paper 1. **Propose a new category of passive loss functions**: The author introduces a new category of passive loss functions, called Normalized Negative Loss Functions (NNLFs). By replacing the existing MAE (Mean Absolute Error), these new functions can focus more effectively on clean samples in memory, thereby accelerating convergence and improving training efficiency. 2. **Construct the Active Negative Loss (ANL) framework**: Based on NNLFs, the author proposes a new loss function framework - Active Negative Loss (ANL), which combines the normalized active loss function and NNLFs, aiming to improve the robustness of the model under noisy labels. 3. **Solve the class imbalance problem in asymmetric noise scenarios**: Aiming at the class imbalance problem that may be caused by asymmetric noisy labels, the author proposes an entropy - based regularization technique, which makes the probability distribution of model outputs more balanced, thereby alleviating the negative impact of class imbalance. 4. **Theoretical proof and experimental verification**: The author not only theoretically proves the robustness of the proposed NNLFs and ANL framework under noisy labels, but also verifies the superior performance of these new loss functions in image classification and segmentation tasks through extensive experiments. ### Formula summary - **Normalized Negative Loss Functions (NNLFs)**: \[ L_{nn}(f(x), y; L)=\frac{1 - A+L(f(x), y)}{\sum_{k = 1}^{K}(A - L(f(x), k))} \] where \(A\) is a constant representing the maximum value of the loss function \(L\). - **Entropy regularization term**: \[ L_{reg}(\pi)=\log K+\sum_{k = 1}^{K}\pi(k)\log\pi(k) \] where \(\pi(k)\) represents the average probability that the model classifies a sample as class \(k\). Through these improvements, the paper provides an effective solution, enabling deep - learning models to maintain good performance in a noisy - label environment and demonstrating its advantages in multiple practical application scenarios.

Active Negative Loss: A Robust Framework for Learning with Noisy Labels

Asymmetric Loss Functions for Learning with Noisy Labels

Noise is the Fatal Poison: A Noise-aware Network for Noisy Dataset Classification

Normalized Loss Functions for Deep Learning with Noisy Labels

Asymmetric Loss Functions for Noise-Tolerant Learning: Theory and Applications

Dynamics-Aware Loss for Learning with Label Noise

Noisy Label Learning with Provable Consistency for a Wider Family of Losses.

Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning

Dynamic Loss For Robust Learning

L_DMI: An Information-theoretic Noise-robust Loss Function

Noise-Robust Loss Functions: Enhancing Bounded Losses for Large-Scale Noisy Data Learning

Learning with Noisy Labels Via Sparse Regularization

Learning with Noisy Labels Via Self-supervised Adversarial Noisy Masking

Learning With Noisy Labels Over Imbalanced Subpopulations

BadLabel: A Robust Perspective on Evaluating and Enhancing Label-noise Learning

Towards Robust Learning with Noisy and Pseudo Labels for Text Classification

L_DMI: A Novel Information-theoretic Loss Function for Training Deep Nets Robust to Label Noise

Improve Noise Tolerance of Robust Loss via Noise-Awareness

Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels