Abstract:Unknown cyber-attacks have appeared constantly. Several anomaly detection techniques based on semi-supervised learning have been proposed to detect these unknown cyber-attacks. Among them, the Denoising Auto-Encoder (DAE) scheme performs better than others in accuracy but is not good enough in precision. This paper proposes a novel two-stage deep learning structure for network flow anomaly detection by combining the models of Gate Recurrent Unit (GRU) and DAE. By using supervised anomaly detection with a selection mechanism to assist semi-supervised anomaly detection, the precision and accuracy of the anomaly detection system are improved. In the proposed structure, we first use the GRU model to analyze the network flow and then take the outcome from the Softmax function as a confidence score. When the score is more than or equal to the predefined confidence threshold, the GRU model outputs the flow as a positive result, no matter the flow is classified as normal or abnormal. When the score is less than the confidence threshold, GRU model outputs the flow as a negative result and passes the flow to DAE model for flow classification. DAE then determines a reconstruction error threshold by learning the pattern of normal flows. Accordingly, the flow is normal or abnormal depending on whether it is under or over the reconstruction error threshold. A comparative experiment is performed using NSL-KDD dataset as benchmark. The results revealed that the precision using the proposed scheme is 0.83% better than DAE. The accuracy using the proposed approach is 90.21%, which is better than Random Forest, Naïve Bayes, One-Dimensional Convolutional Neural Network, two-stage Auto-Encoder, etc. In addition, the proposed approach is also applied to the environment of software defined network (SDN). By adopting our approach in SDN environment, the precision and F-measure are significantly improved.

Network Anomaly Detection Based on Frequent Sub-Graph Mining Approach and Association Analysis

Graph neural network based robust anomaly detection at service level in SDN driven microservice system

A Deep Learning Approach to Detect Anomaly in Software-Defined Network

Anomaly Detection of Network Streams Via Dense Subgraph Discovery

Subtle Anomaly Detection in Dynamic Networks Using Graph Spectra

Deep Learning Based Anomaly Detection Scheme in Software-Defined Networking

A Network Anomaly Detection Algorithm Based on Natural Neighborhood Graph.

LNAD: Towards Lightweight Network Anomaly Detection in Software-Defined Networking

Network-Wide Forwarding Anomaly Detection and Localization in Software Defined Networks

Large-scale IP Network Behavior Anomaly Detection and Identification Using Substructure-Based Approach and Multivariate Time Series Mining

Network-Wide Anomaly Detection Based On Router Connection Relationships

A Novel Data Mining Method for Network Anomaly Detection Based on Transductive Scheme

A Novel Two-Stage Deep Learning Structure for Network Flow Anomaly Detection

An Abnormal Network Traffic Detection Algorithm Based on Big Data Analysis

An Entropy Based Anomaly Traffic Detection Approach in SDN

Software-Defined-Networking-Enabled Traffic Anomaly Detection and Mitigation

Novel Anomaly Intrusion Detection Algorithm Based on Frequent Subgraph Mining

Intranet Anomaly Detection Method Using Flow Mining and Graph Mining

NCGs: Building a Trustworthy Environment to Identify Abnormal Events Based on Network Connection Behavior Analysis.

A comprehensive survey on graph anomaly detection with deep learning

Subgraph Anomaly Detection in Financial Transaction Networks