Abstract:The unsupervised anomaly detection task based on high-dimensional or multidimensional data occupies a very important position in the field of machine learning and industrial applications; especially in the aspect of network security, the anomaly detection of network data is particularly important. The key to anomaly detection is density estimation. Although the methods of dimension reduction and density estimation have made great progress in recent years, most dimension reduction methods are difficult to retain the key information of original data or multidimensional data. Recent studies have shown that the deep autoencoder (DAE) can solve this problem well. In order to improve the performance of unsupervised anomaly detection, we propose an anomaly detection scheme based on a deep autoencoder (DAE) and clustering methods. The deep autoencoder is trained to learn the compressed representation of the input data and then feed it to clustering approach. This scheme makes full use of the advantages of the deep autoencoder (DAE) to generate low-dimensional representation and reconstruction errors for the input high-dimensional or multidimensional data and uses them to reconstruct the input samples. The proposed scheme could eliminate redundant information contained in the data, improve performance of clustering methods in identifying abnormal samples, and reduce the amount of calculation. To verify the effectiveness of the proposed scheme, massive comparison experiments have been conducted with traditional dimension reduction algorithms and clustering methods. The results of experiments demonstrate that, in most cases, the proposed scheme outperforms the traditional dimension reduction algorithms with different clustering methods.

Improving Unsupervised Anomalous Sound Detection Performance of Autoencoder and Its Variant with Pretrained Deep Belief Network

Exploring Large Scale Pre-Trained Models for Robust Machine Anomalous Sound Detection

Explore the Use of Self-supervised Pre-trained Acoustic Features on Disguised Speech Detection

Auto-AD: Autonomous Hyperspectral Anomaly Detection Network Based on Fully Convolutional Autoencoder

Unsupervised Detection of Anomalous Sound based on Deep Learning and the Neyman-Pearson Lemma

Autoencoder with Group-based Decoder and Multi-task Optimization for Anomalous Sound Detection

Improved autoencoder for unsupervised anomaly detection

Improving Anomalous Sound Detection via Low-Rank Adaptation Fine-Tuning of Pre-Trained Audio Models

Memory-augmented Adversarial Autoencoders for Multivariate Time-series Anomaly Detection with Deep Reconstruction and Prediction

Unsupervised Anomalous Sound Detection for Machine Condition Monitoring Using Classification-Based Methods

Transformer-based Autoencoder with ID Constraint for Unsupervised Anomalous Sound Detection

Predictive AutoEncoders Are Context-Aware Unsupervised Anomalous Sound Detectors

Feature Encoding with AutoEncoders for Weakly-supervised Anomaly Detection

Self-Supervision-Augmented Deep Autoencoder for Unsupervised Visual Anomaly Detection

Integrating adversarial training strategies into deep autoencoders: A novel aeroengine anomaly detection framework

A Contrastive Semi-Supervised Learning Framework For Anomaly Sound Detection.

Unseen Noise Estimation Using Separable Deep Auto Encoder for Speech Enhancement

Analysis of Feature Representations for Anomalous Sound Detection

Deep anomaly detection with self-supervised learning and adversarial training

Unsupervised Anomaly Detection Based on Deep Autoencoding and Clustering

Improved AutoEncoder with LSTM module and KL divergence