Abstract:The presence of outliers can severely degrade learned representations and performance of deep learning methods and hence disproportionately affect the training process, leading to incorrect conclusions about the data. For example, anomaly detection using deep generative models is typically only possible when similar anomalies (or outliers) are not present in the training data. Here we focus on variational autoencoders (VAEs). While the VAE is a popular framework for anomaly detection tasks, we observe that the VAE is unable to detect outliers when the training data contains anomalies that have the same distribution as those in test data. In this paper we focus on robustness to outliers in training data in VAE settings using concepts from robust statistics. We propose a variational lower bound that leads to a robust VAE model that has the same computational complexity as the standard VAE and contains a single automatically-adjusted tuning parameter to control the degree of robustness. We present mathematical formulations for robust variational autoencoders (RVAEs) for Bernoulli, Gaussian and categorical variables. The RVAE model is based on beta-divergence rather than the standard Kullback-Leibler (KL) divergence. We demonstrate the performance of our proposed β-divergence-based autoencoder for a variety of image and categorical datasets showing improved robustness to outliers both qualitatively and quantitatively. We also illustrate the use of our robust VAE for detection of lesions in brain images, formulated as an anomaly detection task. Finally, we suggest a method to tune the hyperparameter of RVAE which makes our model completely unsupervised.

Robust Relevance Vector Machine with Noise Variance Coefficient

A Novel Robust Smooth Support Vector Machine

M-estimator Based Support Vector Machine and Its Application

Noise model based ν-support vector regression with its application to short-term wind speed forecasting

A Novel Bayesian Robust Model and Its Application for Fault Detection and Automatic Supervision of Nonlinear Process

A Gaussian Approximation of Marginal Likelihood in Relevance Vector Machine for Industrial Data With Input Noise

Combined modeling of sparse and dense noise for improvement of Relevance Vector Machine

Adaptive sparseness for correntropy-based robust regression via automatic relevance determination

Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis

Heteroscedastic Relevance Vector Machine

Robust LS-SVR Based on Variational Bayesian and Its Applications

Robust noise-aware algorithm for randomized neural network and its convergence properties

A robust variational autoencoder using beta divergence

Robust Estimation for Image Noise Based on Eigenvalue Distributions of Large Sample Covariance Matrices.

On the Robustness of Nearest Neighbor with Noisy Data

Robust support vector machine training via convex outlier ablation

Speech recognition based on relevance vector machine

On a Robust Variogram Estimator

Robust Estimation and Outlier Detection for Varying-Coefficient Models Via Penalized Regression

On Robustness to Missing Video for Audiovisual Speech Recognition

Robust Outlier Rejection for 3D Registration with Variational Bayes