Abstract:Abstract Data distribution presents sparsity in a high-dimensional space, thus difficulty affording sufficient information to distinguish anomalies from normal instances. Moreover, a high-dimensional space may exist many subspaces, obviously, anomalies can exist in any subspaces. This also creates trouble for anomaly mining. Consequently, it is a challenge for anomaly mining in a high-dimensional space. To address this, here proposed a deep hypersphere method fused with probabilistic approach for anomaly mining. In the proposed method, the deep neural network is used as a feature extractor to capture those layered low-dimensional features from the data lying in a high-dimensional space. To promote the ability of the deep neural network to capture these features, the probability approach of sample binary-classification is fused into the loss function, thereby forming the probability deep neural network Then, the hypersphere is used as an anomalous detector. In the low-dimensional features extracted by the deep neural network, the anomalous detector separates anomaly features from normal features. Finally, experimental results on synthetic and real-world data sets show that the proposed method not only outperforms the state-of-the-art methods in the precision of mined anomalies, but also this hybrid method consisting of deep neural networks and traditional detection methods has outstanding capabilities of mining high-dimensional anomalies. We find that deep neural networks fusing the probabilistic method of sample multi-classification can capture these desired low-dimensional features; moreover, these captured low-dimensional features present more obvious layered characteristics. We also demonstrate that as long as these captured features represent a fewer anomaly instances, it can sufficiently identify anomalies from normal instances.

Anomaly detection based on weighted fuzzy-rough density

Exploiting fuzzy rough entropy to detect anomalies

A fuzzy detection approach to high-dimensional anomalies

An Empirical Study on Anomaly Detection Using Density-based and Representative-based Clustering Algorithms

Anomaly Detection with Variance Stabilized Density Estimation

Exploring the Intrinsic Probability Distribution for Hyperspectral Anomaly Detection

Weighted subspace anomaly detection in high-dimensional space

Simulation Research on Anomaly Detection in Large Data Environment

Anomaly Detection with Partially Observed Anomalies

Anomaly Detection Algorithm Based on Subspace Local Density Estimation

Unsupervised Anomaly Detection by Robust Density Estimation

Anomaly detection for high-dimensional space using deep hypersphere fused with probability approach

Characterizing Network Anomaly Traffic with Euclidean Distance-Based Multiscale Fuzzy Entropy.

Clustering-based anomaly detection in multivariate time series data

Importance Weighted Adversarial Discriminative Transfer for Anomaly Detection

A weighted fuzzy C-means clustering method with density peak for anomaly detection in IoT-enabled manufacturing process

Dynamic weight-based granular representation of time series and its application in collective anomaly detection

Title : Adaptive Anomaly Detection using Isolation Forest

Anomaly Detection Algorithm Based on CFSFDP

Ensemble Entropy Metric for Hyperspectral Anomaly Detection

An anomaly detection algorithm for hyperspectral images using subspace sparse representation