Abstract:The unsupervised anomaly detection task based on high-dimensional or multidimensional data occupies a very important position in the field of machine learning and industrial applications; especially in the aspect of network security, the anomaly detection of network data is particularly important. The key to anomaly detection is density estimation. Although the methods of dimension reduction and density estimation have made great progress in recent years, most dimension reduction methods are difficult to retain the key information of original data or multidimensional data. Recent studies have shown that the deep autoencoder (DAE) can solve this problem well. In order to improve the performance of unsupervised anomaly detection, we propose an anomaly detection scheme based on a deep autoencoder (DAE) and clustering methods. The deep autoencoder is trained to learn the compressed representation of the input data and then feed it to clustering approach. This scheme makes full use of the advantages of the deep autoencoder (DAE) to generate low-dimensional representation and reconstruction errors for the input high-dimensional or multidimensional data and uses them to reconstruct the input samples. The proposed scheme could eliminate redundant information contained in the data, improve performance of clustering methods in identifying abnormal samples, and reduce the amount of calculation. To verify the effectiveness of the proposed scheme, massive comparison experiments have been conducted with traditional dimension reduction algorithms and clustering methods. The results of experiments demonstrate that, in most cases, the proposed scheme outperforms the traditional dimension reduction algorithms with different clustering methods.

An Unsupervised Learning Short Text Clustering Method

An Unsupervised Learning Short Text Clustering Method

Deep Structured Clustering of Short Text

Research on Neural Network Clustering Algorithm for Short Text

A Short Text Clustering Approaches in Social Media

Text Clustering on Short Message by Using Deep Semantic Representation

Short text clustering based on word embeddings and EMD

Unsupervised Deep Embedding for Clustering Analysis

Unsupervised Deep Embedding for Fuzzy Clustering

Representation Learning for Short Text Clustering

Self-Taught convolutional neural networks for short text clustering

Unsupervised discriminative feature learning via finding a clustering-friendly embedding space

Text clustering based on pre-trained models and autoencoders

Unsupervised Deep Discriminant Analysis Based Clustering

Pseudo-supervised Deep Subspace Clustering

Self-Supervised Self-Organizing Clustering Network: A Novel Unsupervised Representation Learning Method

Deep subspace clustering to achieve jointly latent feature extraction and discriminative learning

An end-to-end Neural Network Framework for Text Clustering

Clustering and Network Analysis for the Embedding Spaces of Sentences and Sub-Sentences

Unsupervised Anomaly Detection Based on Deep Autoencoding and Clustering

A Novel Text Clustering Algorithm Based on Inner Product Space Model of Semantic