Abstract:Learning with noisy labels has become an effective strategy for enhancing the robustness of models, which enables models to better tolerate inaccurate data. Existing methods either focus on optimizing the loss function to mitigate the interference from noise, or design procedures to detect potential noise and correct errors. However, their effectiveness is often compromised in representation learning due to the dilemma where models overfit to noisy labels. To address this issue, this paper proposes a relation modeling and distillation framework that models inter-sample relationships via self-supervised learning and employs knowledge distillation to enhance understanding of latent associations, which mitigate the impact of noisy labels. Specifically, the proposed method, termed RMDNet, includes two main modules, where the relation modeling (RM) module implements the contrastive learning technique to learn representations of all data, an unsupervised approach that effectively eliminates the interference of noisy tags on feature extraction. The relation-guided representation learning (RGRL) module utilizes inter-sample relation learned from the RM module to calibrate the representation distribution for noisy samples, which is capable of improving the generalization of the model in the inference phase. Notably, the proposed RMDNet is a plug-and-play framework that can integrate multiple methods to its advantage. Extensive experiments were conducted on two datasets, including performance comparison, ablation study, in-depth analysis and case study. The results show that RMDNet can learn discriminative representations for noisy data, which results in superior performance than the existing methods.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to mitigate the impact of noisy labels on model performance when training machine - learning models on datasets with noisy labels. Specifically, existing methods either reduce noise interference by optimizing the loss function or design processes to detect and correct potential mis - labeled labels. However, these methods often perform poorly in representation learning because the model is prone to over - fitting to noisy labels. To address this challenge, the paper proposes a framework based on relationship modeling and knowledge distillation - RMDNet, which aims to guide representation learning by modeling the relationships between samples, thereby reducing the negative impact of noisy labels. ### Main Contributions 1. **Proposed a new framework**: RMDNet, which mitigates the negative impact of noisy labels and improves the robustness of the model by modeling the relationships between samples and using knowledge distillation. 2. **Universal framework**: RMDNet is a universal framework that can be combined with other methods to improve their performance without changing the core architecture. 3. **Revealed errors in representation learning**: The study found that errors in representation learning are one of the main factors affecting the robustness of the model. RMDNet mitigates the interference of noisy labels on representation learning through relationship modeling and distillation to ensure the effectiveness of the framework. ### Method Overview The RMDNet framework contains two main modules: 1. **Relationship Modeling Module (RM Module)**: Utilizes self - supervised learning techniques to extract relationships between sample representations without relying on externally labeled labels, thereby ensuring the quality and reliability of the extracted sample representations. 2. **Relationship - Guided Representation Learning Module (RGRL Module)**: Utilizes the relationship knowledge learned from the RM module to correct the representation distribution of noisy - label samples, improving the generalization ability of the model in the inference stage. ### Key Technologies - **Self - supervised learning**: Uses the SimSiam framework for representation learning to maximize the similarity between two views of the same image without using negative sample pairs and momentum encoders. - **Relationship graph construction**: Constructs a relationship graph by calculating the Pearson Correlation Coefficient (PPC) between each pair of sample representations. - **Knowledge distillation**: Transfers the pre - trained relationship graph knowledge to the task network through edge - matching and node - matching losses, reducing the impact of noisy labels on the task network and improving the accuracy of model training. ### Experimental Results The paper conducted experiments on the CIFAR - 10 and CIFAR - 100 datasets and tested the performance under different noise rates (10%, 20%, 40%) under symmetric noise, asymmetric noise, and flipped noise conditions respectively. The experimental results show that RMDNet outperforms the baseline method and other existing methods under various noise conditions. ### Conclusion RMDNet effectively mitigates the negative impact of noisy labels on model training through relationship modeling and knowledge distillation, improving the robustness and accuracy of the model. The universality and flexibility of this framework enable it to be combined with other methods to further improve performance.

Relation Modeling and Distillation for Learning with Noisy Labels

Learning from Noisy Labels with Decoupled Meta Label Purifier

A Latent-Label Denoising Method for Relation Extraction with Self-Directed Confidence Learning

Reinforcement Learning for Distantly Supervised Relation Extraction

A Noise Adaptive Model for Distantly Supervised Relation Extraction

Online Noisy Continual Relation Learning.

A Two-Level Noise-Tolerant Model for Relation Extraction with Reinforcement Learning.

Reducing Wrong Labels for Distantly Supervised Relation Extraction With Reinforcement Learning

Distantly-Supervised Joint Extraction with Noise-Robust Learning

Exploiting Noisy Data in Distant Supervision Relation Classification

Reinforcement Learning for Relation Classification From Noisy Data

Reinforced Natural Language Inference for Distantly Supervised Relation Classification

A Noisy Label and Negative Sample Robust Loss Function for DNN-based Distant Supervised Relation Extraction

Relation Mention Extraction from Noisy Data with Hierarchical Reinforcement Learning

Countering Noisy Labels by Learning from Auxiliary Clean Labels

ACE: A Coarse-to-Fine Learning Framework for Reliable Representation Learning Against Label Noise

A Fine-grained and Noise-aware Method for Neural Relation Extraction

Distant Supervision for Relation Extraction via Noise Filtering.

A Model-Agnostic Approach for Learning with Noisy Labels of Arbitrary Distributions

Co-LDL: A Co-Training-Based Label Distribution Learning Method for Tackling Label Noise

Label Distribution Learning with Noisy Labels Via Three-Way Decisions