Abstract:Most existing approaches to anomaly detection focus on statistical features of the data. However, in many cases, users are merely interested in a subset of the statistical outliers depending on the specific domain of interest, e.g., network attacks or financial fraud. The instruction from human experts is therefore indispensable in building predictive models in such applications. However, obtaining labels from human experts is time-consuming and expensive. Obtaining labels from nonexpert labelers are relatively easy and cost-effective. However, the labeling accuracy of a nonexpert is usually difficult to assess. Therefore, it remains open to leverage both the machine intelligence and the knowledge from labelers with diverse backgrounds to construct a machine learning model for domain-specific anomaly detection. To this end, this paper proposes a framework of tripartite active learning for interactive anomaly discovery in large datasets based on crowdsourced labels. This tripartite active learning method consists of two stages. In the first stage, an unsupervised learning algorithm is employed to extract statistical outliers from the dataset. This algorithm is of low computational complexity as well as memory requirement and thus well suited for large datasets. We then develop an iterative algorithm consisting of two steps. The algorithm first evaluates and trains labelers based on gold instances provided by the expert labelers. Then, it assigns the most informative samples to its most confident labeler for relabeling and update the detector based on new labels. The capacity constraints are taken into account in the active learning approach to guarantee the fair allocation of labeling instances as well as robustness against erroneous labels. It is seen through experiments that the proposed algorithm provides an effective means for interactive anomaly detection. As far as we are aware of, this is the first work that considers designing a tripartite machine learning system for domain-specific anomaly detection.

ActiveAD: Enhancing Anomaly Detection in Tabular Data through Active Learning Strategies

Deep Anomaly Detection Via Active Anomaly Search.

Data-Efficient and Interpretable Tabular Anomaly Detection

Cross-dataset Time Series Anomaly Detection for Cloud Systems

ATDAD: One-class Adversarial Learning for Tabular Data Anomaly Detection

Tripartite Active Learning for Interactive Anomaly Discovery

Little Help Makes a Big Difference: Leveraging Active Learning to Improve Unsupervised Time Series Anomaly Detection

Implementing Active Learning in Cybersecurity: Detecting Anomalies in Redacted Emails

Deep Reinforced Active Learning for Time Series Anomaly Detection.

Boundary-Driven Active Learning for Anomaly Detection in Time Series Data Streams.

Learning to Detect Interesting Anomalies

Retrieval Augmented Deep Anomaly Detection for Tabular Data

Feature Interaction-Based Reinforcement Learning for Tabular Anomaly Detection

Interactive Context-Aware Anomaly Detection Guided by User Feedback

AFALog: A General Augmentation Framework for Log-based Anomaly Detection with Active Learning

ALAD: A New Unsupervised Time Series Anomaly Detection Paradigm Based on Activation Learning

Effectiveness of Tree-based Ensembles for Anomaly Discovery: Insights, Batch and Streaming Active Learning

Online Adaptive Asymmetric Active Learning for Budgeted Imbalanced Data.

Active anomaly detection based on deep one-class classification

Label-Efficient Interactive Time-Series Anomaly Detection

AD-LLM: Benchmarking Large Language Models for Anomaly Detection