Abstract:Robust regression plays an important role in many machine learning problems. A primal approach relies on the use of Huber loss and an iteratively reweighted ℓ 2 method. However, because the Huber loss is not smooth and its corresponding distribution cannot be represented as a Gaussian scale mixture, such an approach is extremely difficult to handle using a probabilistic framework. To address those limitations, this paper proposes two novel losses and the corresponding probability functions. One is called Soft Huber , which is well suited for modeling non-Gaussian noise. Another is Nonconvex Huber , which can help produce much sparser results when imposed as a prior on regression vector. They can represent any ℓ q loss ( 1 2 ⩽ q < 2) with tuning parameters, which makes the regression model more robust. We also show that both distributions have an elegant form, which is a Gaussian scale mixture with a generalized inverse Gaussian mixing density. This enables us to devise an expectation maximization (EM) algorithm for solving the regression model. We can obtain an adaptive weight through EM, which is very useful to remove noise data or irrelevant features in regression problems. We apply our model to the face recognition problem and show that it not only reduces the impact of noise pixels but also removes more irrelevant face images. Our experiments demonstrate the promising results on two datasets.

Adaptively Weighted PCA Algorithm

Robust Principal Component Analysis Based on Maximum Correntropy Criterion

Adaptive Kpca Modeling of Nonlinear Systems

Normalized Robust PCA With Adaptive Reconstruction Error Minimization

Robust Principal Component Analysis via Discriminant Sample Weight Learning

Improved Algorithms for High-Dimensional Robust Pca

Adaptive Weighted Sparse Principal Component Analysis for Robust Unsupervised Feature Selection

Sample Weighting: an Inherent Approach for Outlier Suppressing Discriminant Analysis

An Iterative Algorithm for Robust Kernel Principal Component Analysis

Avoiding Optimal Mean Robust PCA/2DPCA with Non-Greedy L1-norm Maximization

Robust and Sparse Principal Component Analysis With Adaptive Loss Minimization for Feature Selection

Robust generalized PCA for enhancing discriminability and recoverability

Avoiding Optimal Mean ℓ 2,1 -Norm Maximization-Based Robust PCA for Reconstruction

Robust Sparse PCA Via Weighted Elastic Net

Dynamically optimizing face recognition using PCA

Self-Paced Probabilistic Principal Component Analysis for Data with Outliers

Robust principal component analysis for sparse face recognition

Probabilistic Robust Regression with Adaptive Weights — a Case Study on Face Recognition

Sparse PCA: Optimal rates and adaptive estimation