A fuzzy detection approach to high-dimensional anomalies

Jian Zheng,Nanshan Ruan,Pingping Wei,Lin Li,Jingyue Zhang
DOI: https://doi.org/10.1007/s00530-024-01343-7
IF: 3.9
2024-05-16
Multimedia Systems
Abstract:Rare anomalies allow to be hidden in any subspace upon a high-dimensional space so that high-dimensional dimensional of the data brings a lot of trouble for anomalous detectors. To mine high-dimensional anomalies, this paper proposes a novel hypersphere with density fuzzy. On the one hand, the major contributors are chosen by using the fuzzy and a density estimator to quantify the contributions created by unknown instances, and then the hypersphere trained by the major contributors achieves anomalous identification. On the other hand, an inner product kernel is also derived to assist that the hypersphere pays more attention to local regions containing anomalies. Experimental results on ten UCI datasets show that the proposed model wins over the opponents on the three ultra-high dimensional datasets and most high-dimensional datasets in mining anomalies. Results also indicate that these models with fuzzy perform better than those without fuzzy, meanwhile, there is no significant difference between detected results. We demonstrate that calculating data density or attending data local regions can alleviate negative effects caused by high dimensionality on anomalous detection results to a certain extent. Additionally, the effort created by fuzzy to resist the curse of dimensionality do not rely on specific scenarios and specific detectors, while the assistance afforded by kernels to resist high dimensionality is dependent of detector structures.
computer science, information systems, theory & methods
What problem does this paper attempt to address?