Abstract:The unsupervised anomaly detection task based on high-dimensional or multidimensional data occupies a very important position in the field of machine learning and industrial applications; especially in the aspect of network security, the anomaly detection of network data is particularly important. The key to anomaly detection is density estimation. Although the methods of dimension reduction and density estimation have made great progress in recent years, most dimension reduction methods are difficult to retain the key information of original data or multidimensional data. Recent studies have shown that the deep autoencoder (DAE) can solve this problem well. In order to improve the performance of unsupervised anomaly detection, we propose an anomaly detection scheme based on a deep autoencoder (DAE) and clustering methods. The deep autoencoder is trained to learn the compressed representation of the input data and then feed it to clustering approach. This scheme makes full use of the advantages of the deep autoencoder (DAE) to generate low-dimensional representation and reconstruction errors for the input high-dimensional or multidimensional data and uses them to reconstruct the input samples. The proposed scheme could eliminate redundant information contained in the data, improve performance of clustering methods in identifying abnormal samples, and reduce the amount of calculation. To verify the effectiveness of the proposed scheme, massive comparison experiments have been conducted with traditional dimension reduction algorithms and clustering methods. The results of experiments demonstrate that, in most cases, the proposed scheme outperforms the traditional dimension reduction algorithms with different clustering methods.

Identifying Anomaly Aircraft Trajectories in Terminal Areas Based on Deep Auto?encoder and Its Application in Trajectory Clustering

Aircraft Trajectory Clustering in Terminal Airspace Based on Deep Autoencoder and Gaussian Mixture Model

A Deep Trajectory Clustering Method Based on Sequence‐to‐sequence Autoencoder Model

Anomalous State Recognition of Lane-changing Behavior Using a Hybrid Autoencoder Architecture

Spatial-Temporal Graph Discriminant AutoEncoder for Traffic Congestion Forecasting

A Clustering Ensemble Method of Aircraft Trajectory Based on the Similarity Matrix

DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation

Detection of Anomalous Traffic Patterns and Insight Analysis from Bus Trajectory Data

Real-Time Detection of Anomalous Taxi Trajectories from GPS Traces.

Helicopter Track Identification with Autoencoder

Unsupervised learning trajectory anomaly detection algorithm based on deep representation

Unsupervised Anomaly Detection Based on Deep Autoencoding and Clustering

An Aircraft Trajectory Prediction Method Based on Trajectory Clustering and a Spatiotemporal Feature Network

Deep learning detection of anomalous patterns from bus trajectories for traffic insight analysis

Road User Abnormal Trajectory Detection using a Deep Autoencoder

Iboat: Isolation-Based Online Anomalous Trajectory Detection

A Flight Trajectory Prediction Method based on Trajectory Clustering

Autoencoding Features for Aviation Machine Learning Problems

Clustering Approach for Trajectory Anomaly Detection

Industrial Control Malicious Traffic Anomaly Detection System Based on Deep Autoencoder

Automated data-driven prediction on aircraft Estimated Time of Arrival