Abstract:Mixed-type data with both categorical and numerical features are ubiquitous in network security, but the existing methods are minimal to deal with them. Existing methods usually process mixed-type data through feature conversion, whereas their performance is downgraded by information loss and noise caused by the transformation. Meanwhile, existing methods usually superimpose domain knowledge and machine learning in which fixed thresholds are used. It cannot dynamically adjust the anomaly threshold to the actual scenario, resulting in inaccurate anomalies obtained, which results in poor performance. To address these issues, this paper proposes a novel Anomaly Detection method based on Reinforcement Learning, termed ADRL, which uses reinforcement learning to dynamically search for thresholds and accurately obtain anomaly candidate sets, fusing domain knowledge and machine learning fully and promoting each other. Specifically, ADRL uses prior domain knowledge to label known anomalies and uses entropy and deep autoencoder in the categorical and numerical feature spaces, respectively, to obtain anomaly scores combining with known anomaly information, which are integrated to get the overall anomaly scores via a dynamic integration strategy. To obtain accurate anomaly candidate sets, ADRL uses reinforcement learning to search for the best threshold. Detailedly, it initializes the anomaly threshold to get the initial anomaly candidate set and carries on the frequent rule mining to the anomaly candidate set to form the new knowledge. Then, ADRL uses the obtained knowledge to adjust the anomaly score and get the score modification rate. According to the modification rate, different threshold modification strategies are executed, and the best threshold, that is, the threshold under the maximum modification rate, is finally obtained, and the modified anomaly scores are obtained. The scores are used to re-carry out machine learning to improve the algorithm's accuracy for anomalous data. Repeat the above process until the method is stable. We experiment on ten real network traffic datasets. Experiments show ADRL averagely improves ROC-AUC and PR-AUC than eight state-of-the-art competitors by 89.6% and 286.0%, respectively.

A Light-weight Online Learning Framework for Network Traffic Abnormality Detection

Network Abnormal Traffic Detection Framework Based on Deep Reinforcement Learning

Enhanced Anomaly Traffic Detection Framework Using BiGAN and Contrastive Learning

Detecting Traffic Anomaly in Wireless Networks, an Analytics Methodology

A Network Traffic Anomaly Detection Method Based on Gaussian Mixture Model

Network Abnormal Traffic Detection Model Based on Semi-Supervised Deep Reinforcement Learning

Abnormal Traffic Detection: Traffic Feature Extraction and DAE-GAN With Efficient Data Augmentation

A Novel Multi-Modal Incremental Tensor Decomposition for Anomaly Detection in Large-Scale Networks

Abnormal Traffic Detection Based on Generative Adversarial Network and Feature Optimization Selection

Anomaly Detection in Event-triggered Traffic Time Series Via Similarity Learning

A Semi-Supervised Anomaly Network Traffic Detection Framework Via Multimodal Traffic Information Fusion

Dynamic Group Behavior Analysis and Its Application in Network Abnormal Behavior Detection.

A Traffic Anomaly Detection and Identification Approach Based on Multi-instance Learning.

Research on Network Security Filtering Model and Key Algorithms Based on Network Abnormal Traffic Analysis

GSOOA-1DDRSN: Network traffic anomaly detection based on deep residual shrinkage networks

A Lightweight Bit-Operation Abnormal Traffic Detection Method Based On XNOR-CNN.

An Adaptive Profile-Based Approach for Detecting Anomalous Traffic in Backbone.

Effective Anomaly Detection Based on Reinforcement Learning in Network Traffic Data.

Pattern Discovery and Multi-Slot-Ahead Forecast of Network Traffic: A Revisiting to Gaussian Process

Anomalous Trafifc Detection Based on Trafifc Behavior Characteristics

Towards A Multi-Layers Anomaly Detection Framework for Analyzing Network Traffic