Abstract:We propose a novel Conditional Latent space Variational Autoencoder (CL-VAE) to perform improved pre-processing for anomaly detection on data with known inlier classes and unknown outlier classes. This proposed variational autoencoder (VAE) improves latent space separation by conditioning on information within the data. The method fits a unique prior distribution to each class in the dataset, effectively expanding the classic prior distribution for VAEs to include a Gaussian mixture model. An ensemble of these VAEs are merged in the latent spaces to form a group consensus that greatly improves the accuracy of anomaly detection across data sets. Our approach is compared against the capabilities of a typical VAE, a CNN, and a PCA, with regards AUC for anomaly detection. The proposed model shows increased accuracy in anomaly detection, achieving an AUC of 97.4% on the MNIST dataset compared to 95.7% for the second best model. In addition, the CL-VAE shows increased benefits from ensembling, a more interpretable latent space, and an increased ability to learn patterns in complex data with limited model sizes.

What problem does this paper attempt to address?

The paper attempts to address the problem of more effective anomaly detection on datasets with known normal categories but unknown anomaly categories. Specifically, the authors propose a novel Conditional Latent Space Variational Autoencoder (CL-VAE) that improves latent space separation by conditioning information in the data, thereby enhancing the accuracy of anomaly detection. This method extends the prior distribution in the classical Variational Autoencoder (VAE) to include a Gaussian Mixture Model by fitting a unique prior distribution for each category. Additionally, by merging multiple CL-VAE models in the latent space to form a group consensus, the accuracy of anomaly detection is further improved. The main contributions of the paper include: 1. **Multiple Latent Gaussian Distributions**: By abandoning the traditional single Gaussian assumption, it allows different categories to have different distributions, reducing category overlap and confusion. 2. **Radial Latent Space Separation**: By fixing cluster centers on a circumference, it forces the latent space center to be empty, making it easier for anomaly points to cluster together, thereby better separating anomaly points from normal points. 3. **Latent Space Integration**: By using multiple encoders to form multiple latent spaces and merging these spaces into a group consensus space, the model's prediction accuracy and robustness are further enhanced. The paper demonstrates the superiority of CL-VAE in anomaly detection by comparing its performance with traditional VAE, Convolutional Neural Networks (CNN), and Principal Component Analysis (PCA) on multiple datasets. Specifically, CL-VAE achieved 97.4% AUC on the MNIST dataset, while the second-best model only achieved 95.7%. Additionally, CL-VAE also shows characteristics of benefiting more from integration, having more interpretable latent spaces, and being more capable of learning patterns in complex data.

Improved Anomaly Detection through Conditional Latent Space VAE Ensembles

VAE-based Deep SVDD for anomaly detection

VELC: A New Variational AutoEncoder Based Model for Time Series Anomaly Detection

Time Series Anomaly Detection with Variational Autoencoders

Multi-Channel Multi-Scale Convolution Attention Variational Autoencoder (MCA-VAE): An Interpretable Anomaly Detection Algorithm Based on Variational Autoencoder

Variational Autoencoder for Anomaly Detection: A Comparative Study

Variational Auto Encoder Fused with Gaussian Process for Unsupervised Anomaly Detection

Anomaly Detection for Time Series Using VAE-LSTM Hybrid Model

VESC: a new variational autoencoder based model for anomaly detection

Improved AutoEncoder with LSTM module and KL divergence

Latent Vector Expansion using Autoencoder for Anomaly Detection

Conditional out-of-sample generation for unpaired data using trVAE

Convolutional autoencoder based on latent subspace projection for anomaly detection

Disentangled Anomaly Detection for Multivariate Time Series

Anomaly Detection of Time Series With Smoothness-Inducing Sequential Variational Auto-Encoder

Spatial-contextual variational autoencoder with attention correction for anomaly detection in retinal OCT images

Harnessing Feature Clustering For Enhanced Anomaly Detection With Variational Autoencoder And Dynamic Threshold

Multi-mode non-Gaussian variational autoencoder network with missing sources for anomaly detection of complex electromechanical equipment

NVAE-GAN Based Approach for Unsupervised Time Series Anomaly Detection

Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspective

A Non-Parametric Subspace Analysis Approach with Application to Anomaly Detection Ensembles