Abstract:Validating the contributions of new algorithms is a critical step in hyperspectral anomaly detection (HAD) research. Typically, validation involves comparing the performance of a proposed algorithm against other algorithms using a series of benchmark datasets. Despite the longstanding use of this comparison process, little attention has been paid to the characteristics of datasets and algorithms that ensure each algorithm has an equal opportunity of performing well. Characteristics of datasets and algorithms that inadvertently favor one algorithm can skew results, leading to misleading conclusions. To address this issue, this study introduces a feature-centric framework designed to assist in ensuring an unbiased comparison of HAD algorithms. The framework identifies significant correlations between datasets and algorithms by extracting distribution-related features from the datasets and statistically testing them against the algorithmic outcomes. The identified trends are then compared across datasets to ensure that all relevant trends are equally represented, thereby ensuring diversity and validating that no singular algorithm is afforded an inherent advantage. The framework was tested on five algorithms across 14 datasets. The results indicate that multiple measures of variance within the datasets are key drivers of diversity, and these measures accurately predicted algorithmic outcomes for 12 of the 14 datasets. This suggests that the identified trends effectively explain the algorithmic outcomes and highlights the importance of incorporating datasets with a diverse range of variances in comparisons of HAD algorithms.

On Equivalence of Anomaly Detection Algorithms

Experimental Comparison and Survey of Twelve Time Series Anomaly Detection Algorithms

Local Evaluation of Time Series Anomaly Detection Algorithms

Uncertainty-aware Evaluation of Auxiliary Anomalies with the Expected Anomaly Posterior

gen2Out: Detecting and Ranking Generalized Anomalies

Sparse Anomaly Detection Across Referentials: A Rank-Based Higher Criticism Approach

Exploring the Impact of Outlier Variability on Anomaly Detection Evaluation Metrics

Unsupervised anomaly detection algorithms on real-world data: how many do we need?

Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark

Anomaly Detection with Partially Observed Anomalies

EnsembleLens: Ensemble-based Visual Exploration of Anomaly Detection Algorithms with Multidimensional Data

Functional Anomaly Detection: a Benchmark Study

Anomaly Detection Based on Convex Analysis: A Survey

Exploring Global and Local Information for Anomaly Detection with Normal Samples

A Mallows-like Criterion for Anomaly Detection with Random Forest Implementation

Anomaly Detection Based on Isolation Mechanisms: A Survey

Anomaly Detection: How to Artificially Increase your F1-Score with a Biased Evaluation Protocol

Benchmarking Anomaly Detection Methods: Insights From the UCR Time Series Anomaly Archive

Enhancing Hyperspectral Anomaly Detection Algorithm Comparisons: Leveraging Dataset and Algorithm Characteristics

Anomaly Detection via Neighbourhood Contrast.

Learning Competitive and Discriminative Reconstructions for Anomaly Detection