Abstract:Feature noise and label noise are ubiquitous in practical scenarios, which pose great challenges for training a robust machine learning model. Most previous approaches usually deal with only a single problem of either feature noise or label noise. However, in real-world applications, hybrid noise, which contains both feature noise and label noise, is very common due to the unreliable data collection and annotation processes. Although some results have been achieved by a few representation learning based attempts, this issue is still far from being addressed with promising performance and guaranteed theoretical analyses. To address the challenge, we propose a novel unified learning framework called "Feature and Label Recovery" (FLR) to combat the hybrid noise from the perspective of data recovery, where we concurrently reconstruct both the feature matrix and the label matrix of input data. Specifically, the clean feature matrix is discovered by the low-rank approximation, and the ground-truth label matrix is embedded based on the recovered features with a nuclear norm regularization. Meanwhile, the feature noise and label noise are characterized by their respective adaptive matrix norms to satisfy the corresponding maximum likelihood. As this framework leads to a non-convex optimization problem, we develop the non-convex Alternating Direction Method of Multipliers (ADMM) with the convergence guarantee to solve our learning objective. We also provide the theoretical analysis to show that the generalization error of FLR can be upper-bounded in the presence of hybrid noise. Experimental results on several typical benchmark datasets clearly demonstrate the superiority of our proposed method over the state-of-the-art robust learning approaches for various noises.

A probabilistic learning algorithm for robust modeling using neural networks with random weights

A Local Learning Algorithm for Random Weights Networks

Robust noise-aware algorithm for randomized neural network and its convergence properties

Robust Stochastic Configuration Networks with Kernel Density Estimation for Uncertain Data Regression

Probabilistic Robust Regression with Adaptive Weights — a Case Study on Face Recognition

Robust Stochastic Configuration Networks with Maximum Correntropy Criterion for Uncertain Data Regression.

Building Feedforward Neural Networks with Random Weights for Large Scale Datasets

An Iterative Learning Algorithm for Feedforward Neural Networks with Random Weights

A probabilistic framework for learning robust common spatial patterns.

Robust Training of Artificial Feedforward Neural Networks

Robust Nonparametric Regression with Deep Neural Networks

Robust Elastic Net Regression

Multiplicative Reweighting for Robust Neural Network Optimization

Low-Rank-Sparse Subspace Representation for Robust Regression

Randomized Mixture Models for Probability Density Approximation and Estimation

Robust Learning under Hybrid Noise

Robust nonparametric regression based on deep ReLU neural networks

Robust sparse regression by modeling noise as a mixture of gaussians

Robustness of Neural Networks: A Probabilistic and Practical Approach

A Robust Mixed Error Coding Method Based on Nonconvex Sparse Representation

A robust elastic net approach for feature learning