Abstract:The complexity and scale of IT systems are increasing dramatically, posing many challenges to real-world anomaly detection. Deep learning anomaly detection has emerged, aiming at feature learning and anomaly scoring, which has gained tremendous success. However, little work has been done on the thresholding problem despite it being a critical factor for the effectiveness of anomaly detection. In this paper, we model thresholding in anomaly detection as a Markov Decision Process and propose an agent-based dynamic thresholding (ADT) framework based on a deep Q-network. The proposed method can be integrated into many systems that require dynamic thresholding. An auto-encoder is utilized in this study to obtain feature representations and produce anomaly scores for complex input data. ADT can adjust thresholds adaptively by utilizing the anomaly scores from the auto-encoder and significantly improve anomaly detection performance. The properties of ADT are studied through experiments on three real-world datasets and compared with benchmarks, hence demonstrating its thresholding capability, data-efficient learning, stability, and robustness. Our study validates the effectiveness of reinforcement learning in optimal thresholding control in anomaly detection.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **In anomaly detection, how to dynamically adjust the threshold to improve detection performance**. ### Problem Background With the sharp increase in the complexity and scale of IT systems, anomaly detection in the real world faces many challenges. Deep learning has achieved great success in feature learning and anomaly scoring, but when it comes to the threshold setting problem, although it is crucial to the effectiveness of anomaly detection, few studies have been involved. Traditional static or expert - defined threshold methods are difficult to adapt to non - stationary and evolving time - series data, resulting in poor detection performance. ### Paper Objectives The paper aims to solve this problem by introducing an Agent - based Dynamic Threshold (ADT) framework. Specifically, the author models the threshold setting problem as a Markov Decision Process (MDP) and uses Deep Q - Network (DQN) to achieve dynamic threshold adjustment. By combining the anomaly scores generated by Autoencoder (AE), ADT can adaptively adjust the threshold, thereby significantly improving the performance of anomaly detection. ### Key Contributions 1. **Modeling and Framework Proposal**: Model the threshold setting problem in anomaly detection as MDP and propose an Agent - based Dynamic Threshold (ADT) framework. 2. **Experimental Verification**: Experiments were carried out on three real - world benchmark datasets, demonstrating the superiority of ADT in terms of threshold setting, data efficiency, stability, and robustness. 3. **Application Prospects**: Show the application potential of reinforcement learning in optimal threshold control, especially the advantages in dealing with high - dimensional continuous inputs. ### Main Content Overview - **Problem Modeling**: Formalize the threshold setting problem in anomaly detection as MDP, where the state space includes information such as the mean and variance of anomaly scores, the action is binary threshold selection in the space, and the reward function is adjusted according to the detection results. - **Method Design**: Use AE to extract features and generate anomaly scores, and train agents through DQN to dynamically adjust the threshold. - **Experimental Evaluation**: Experiments were carried out on three datasets, namely Yahoo A1Benchmark, SWaT, and WADI, verifying the superiority of ADT over static threshold and other dynamic threshold methods. Through these works, the paper shows the potential of dynamic threshold adjustment in improving the performance of anomaly detection and provides new ideas and methods for future research.

ADT: Agent-based Dynamic Thresholding for Anomaly Detection

Agent-based dynamic thresholding for adaptive anomaly detection using reinforcement learning

An Unsupervised Framework for Anomaly Detection in a Water Treatment System

Dynamic Threshold-based Two-layer Online Unsupervised Anomaly Detector

Accurate Anomaly Detection Leveraging Knowledge-enhanced GAT

Segmentation-Based Adversarial Denoising Auto-Encoder for Anomaly Detection in Multivariate Time Series Data

Hierarchical Context Representation and Self-adaptive Thresholding for Multivariate Anomaly Detection

A Diffusion-Based Framework for Multi-Class Anomaly Detection

ADA: Adaptive Deep Log Anomaly Detector

ADTCD: An Adaptive Anomaly Detection Approach Toward Concept Drift in IoT

An Attention-Based ConvLSTM Autoencoder with Dynamic Thresholding for Unsupervised Anomaly Detection in Multivariate Time Series

Deep Learning-based Anomaly Detection in Cyber-physical Systems

Real-Time Deep Anomaly Detection Framework for Multivariate Time-Series Data in Industrial IoT

TAAD: Time-varying adversarial anomaly detection in dynamic graphs

DTAAD: Dual Tcn-attention networks for anomaly detection in multivariate time series data

A Real-time Anomaly Detection Using Convolutional Autoencoder with Dynamic Threshold

Interactive Learning for Network Anomaly Monitoring and Detection with Human Guidance in the Loop

Research of Anomaly detection based on Dynamic Anomaly Detection Enhancement Framework

AD-NEV: A Scalable Multi-level Neuroevolution Framework for Multivariate Anomaly Detection

Adaptive Anomaly Detection in Network Flows with Low-Rank Tensor Decompositions and Deep Unrolling