Abstract:Mixed-type data with both categorical and numerical features are ubiquitous in network security, but the existing methods are minimal to deal with them. Existing methods usually process mixed-type data through feature conversion, whereas their performance is downgraded by information loss and noise caused by the transformation. Meanwhile, existing methods usually superimpose domain knowledge and machine learning in which fixed thresholds are used. It cannot dynamically adjust the anomaly threshold to the actual scenario, resulting in inaccurate anomalies obtained, which results in poor performance. To address these issues, this paper proposes a novel Anomaly Detection method based on Reinforcement Learning, termed ADRL, which uses reinforcement learning to dynamically search for thresholds and accurately obtain anomaly candidate sets, fusing domain knowledge and machine learning fully and promoting each other. Specifically, ADRL uses prior domain knowledge to label known anomalies and uses entropy and deep autoencoder in the categorical and numerical feature spaces, respectively, to obtain anomaly scores combining with known anomaly information, which are integrated to get the overall anomaly scores via a dynamic integration strategy. To obtain accurate anomaly candidate sets, ADRL uses reinforcement learning to search for the best threshold. Detailedly, it initializes the anomaly threshold to get the initial anomaly candidate set and carries on the frequent rule mining to the anomaly candidate set to form the new knowledge. Then, ADRL uses the obtained knowledge to adjust the anomaly score and get the score modification rate. According to the modification rate, different threshold modification strategies are executed, and the best threshold, that is, the threshold under the maximum modification rate, is finally obtained, and the modified anomaly scores are obtained. The scores are used to re-carry out machine learning to improve the algorithm's accuracy for anomalous data. Repeat the above process until the method is stable. We experiment on ten real network traffic datasets. Experiments show ADRL averagely improves ROC-AUC and PR-AUC than eight state-of-the-art competitors by 89.6% and 286.0%, respectively.

Deep Anomaly Detection and Search via Reinforcement Learning

Deep Anomaly Detection and Search Via Reinforcement Learning (student Abstract)

Deep Anomaly Detection Via Active Anomaly Search.

Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Data

Detecting Anomalies In Encrypted Traffic Via Deep Dictionary Learning

A Deep Reinforcement Learning Method for Accurate and Efficient Anomaly Detection

Spiking Reinforcement Learning for Weakly-Supervised Anomaly Detection

Deep Reinforced Active Learning for Time Series Anomaly Detection.

Deep anomaly detection with self-supervised learning and adversarial training

ESAD: End-to-end Deep Semi-supervised Anomaly Detection

Semi-supervised Anomaly Detection via Adaptive Reinforcement Learning-Enabled Method with Causal Inference for Sensor Signals

ALAD: A New Unsupervised Time Series Anomaly Detection Paradigm Based on Activation Learning

Towards Anomaly Detection in Reinforcement Learning

Semi-supervised learning via DQN for log anomaly detection

Feature Interaction-Based Reinforcement Learning for Tabular Anomaly Detection

Deep Reinforcement Learning for Anomaly Detection: A Systematic Review

Effective Anomaly Detection Based on Reinforcement Learning in Network Traffic Data.

Dually Encoded Semi-supervised Anomaly Detection

Deep Learning for Anomaly Detection: A Review

AnoRand: A Semi Supervised Deep Learning Anomaly Detection Method by Random Labeling

Learning Discriminative Features for Semi-Supervised Anomaly Detection.