Abstract:To cleanse mislabeled examples from a training dataset for efficient and effective induction, most existing approaches adopt a major set oriented scheme: the training dataset is separated into two parts (a major set and a minor set). The classifiers learned from the major set are used to identify noise in the minor set. The obvious drawbacks of such a scheme are twofold: (1) when the underlying data volume keeps growing, it would be either physically impossible or time consuming to load the major set into the memory for inductive learning; and (2) for multiple or distributed datasets, it can be either technically infeasible or factitiously forbidden to download data from other sites (for security or privacy reasons). Therefore, these approaches have severe limitations in conducting effective global data cleansing from large, distributed datasets.In this paper, we propose a solution to bridge the local and global analysis for noise cleansing. More specifically, the proposed effort tries to identify and eliminate mislabeled data items from large or distributed datasets through local analysis and global incorporation. For this purpose, we make use of distributed datasets or partition a large dataset into subsets, each of which is regarded as a local subset and is small enough to be processed by an induction algorithm at one time to construct a local model for noise identification. We construct good rules from each subset, and use the good rules to evaluate the whole dataset. For a given instance I-k , two error count variables are used to count the number of times it has been identified as noise by all data subsets. The instance with higher error values will have a higher probability of being a mislabeled example. Two threshold schemes, majority and non-objection, are used to identify and eliminate the noisy examples. Experimental results and comparative studies on both real-world and synthetic datasets are reported to evaluate the effectiveness and efficiency of the proposed approach.

Data Cleansing for Salt Dome Dataset with Noise Robust Network on Segmentation Task

OT Cleaner: Label Correction As Optimal Transport

Noise is the Fatal Poison: A Noise-aware Network for Noisy Dataset Classification

A Label Noise Robust Stacked Auto-Encoder Algorithm for Inaccurate Supervised Classification Problems

Learning with Noisy Labels Via Self-supervised Adversarial Noisy Masking

Deep Self-Cleansing for Medical Image Segmentation with Noisy Labels

Self-Attention Fully Convolutional DenseNets for Automatic Salt Segmentation

Clean Label Disentangling for Medical Image Segmentation with Noisy Labels

Deep Learning-based Model for Automatic Salt Rock Segmentation

Semantic Segmentation of Airborne LiDAR Point Clouds With Noisy Labels

Learning to detect cloud and snow in remote sensing images from noisy labels

Learning to Segment from Noisy Annotations: A Spatial Correction Approach

Noisy Supervised Deep Learning for Remote Sensing Image Segmentation Using Electronic Maps

Robust Image Classification with Noisy Labels by Negative Learning and Feature Space Renormalization

Learning with Noisy Class Labels for Instance Segmentation

Task-Specific Loss for Robust Instance Segmentation with Noisy Class Labels

A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions From CT Images

Bridging Local and Global Data Cleansing: Identifying Class Noise in Large, Distributed Data Datasets

Sample Prior Guided Robust Model Learning to Suppress Noisy Labels

DAT: Training Deep Networks Robust to Label-Noise by Matching the Feature Distributions

A Benchmark of Long-tailed Instance Segmentation with Noisy Labels