Abstract:Feature noise and label noise are ubiquitous in practical scenarios, which pose great challenges for training a robust machine learning model. Most previous approaches usually deal with only a single problem of either feature noise or label noise. However, in real-world applications, hybrid noise, which contains both feature noise and label noise, is very common due to the unreliable data collection and annotation processes. Although some results have been achieved by a few representation learning based attempts, this issue is still far from being addressed with promising performance and guaranteed theoretical analyses. To address the challenge, we propose a novel unified learning framework called "Feature and Label Recovery" (FLR) to combat the hybrid noise from the perspective of data recovery, where we concurrently reconstruct both the feature matrix and the label matrix of input data. Specifically, the clean feature matrix is discovered by the low-rank approximation, and the ground-truth label matrix is embedded based on the recovered features with a nuclear norm regularization. Meanwhile, the feature noise and label noise are characterized by their respective adaptive matrix norms to satisfy the corresponding maximum likelihood. As this framework leads to a non-convex optimization problem, we develop the non-convex Alternating Direction Method of Multipliers (ADMM) with the convergence guarantee to solve our learning objective. We also provide the theoretical analysis to show that the generalization error of FLR can be upper-bounded in the presence of hybrid noise. Experimental results on several typical benchmark datasets clearly demonstrate the superiority of our proposed method over the state-of-the-art robust learning approaches for various noises.

On the Robustness of Noisy ACE Algorithm and Multi-Layer Residual Learning.

Asynchronous Byzantine-Robust Stochastic Aggregation with Variance Reduction for Distributed Learning

On the Sample Complexity of HGR Maximal Correlation Functions for Large Datasets

Efficient and Robust Deep Learning with Correntropy-induced Loss Function

An Efficient Algorithm for Information Decomposition and Extraction.

An Improved Multi-Branch Residual Network Based on Random Multiplier and Adaptive Cosine Learning Rate Method.

Robust Multi-Layer Extreme Learning Machine Using Bias-Variance Tradeoff

Robust noise-aware algorithm for randomized neural network and its convergence properties

Adaptive sparseness for correntropy-based robust regression via automatic relevance determination

Robust multitask learning in high dimensions under memory constraint

Robust Learning under Hybrid Noise

Robust Regression with Ensembles Communicating over Noisy Channels

Probabilistic Robust Regression with Adaptive Weights — a Case Study on Face Recognition

Generalized Correntropy Based Deep Learning in Presence of Non-Gaussian Noises.

Robust Online Learning over Networks

Huber Loss Based Distributed Robust Learning Algorithm for Random Vector Functional-Link Network

New Robust Metric Learning Model Using Maximum Correntropy Criterion

Local Competition and Uncertainty for Adversarial Robustness in Deep Learning

Evolved-Cooperative Correntropy-Based Extreme Learning Machine for Robust Prediction.

Taking a Moment for Distributional Robustness

Robust Stochastic Configuration Networks with Maximum Correntropy Criterion for Uncertain Data Regression.