Abstract:The Botnets have become one of the most serious threats to cyber infrastructure. Most existing work on detecting botnets is based on flow-based traffic analysis by mining their communication patterns. There also exists related work based on anomaly detection in communication graphs. As bots have continuously evolved and become increasingly sophisticated, only using flow-based traffic analysis or graph-based analysis for the detection would result in false negatives or false positives, or can even be evaded. In this work, we propose BotMark, an automated model that detects botnets with hybrid analysis of flow-based and graph-based network traffic behaviors. We extract 15 statistical flow-based traffic features as well as 3 graph-based features in building the detection model. For flow-based detection, we consider the similarity and stability of C-flow as measurements in the detection. In particular, we employ k-means to measure the similarity of C-flows and assign similarity scores, and calculate stability score of C-flows through the distribution of packet length within a C-flow. The graph-based detection is based on the observation that the neighborhoods of anomalous nodes significantly differ from those of normal nodes in communication graphs. In particular, we use least-square technique and Local Outlier Factor (LOF) to calculate anomaly scores that measure the differences of their neighborhoods. Our models use the scores to mark bots. BotMark performs automated botnet detection with hybrid analysis of flow-based and graph-based traffic behaviors by ensemble of the detection results based on similarity scores, stability scores and anomaly scores. We collect a very large size of network traffic by simulating 5 newly propagated botnets, including Mirai, Black energy, Zeus, Athena and Ares in a real computing environment. Extensive experimental results demonstrate the effectiveness of BotMark. It achieves 99.94% in terms of detection accuracy, outperforming any individual detector with flow-based detection or graph-based detection.

Remote access trojan traffic early detection method based on Markov matrices and deep learning

Low-rate DoS Attack Detection Method Based on Hybrid Deep Neural Networks.

Research of Remote Access Trojan Early Detection Method Using Sequence Analysis

A Highly Efficient Remote Access Trojan Detection Method

Detecting Remote Access Trojans Through External Control at Area Network Borders

PRATD: A Phased Remote Access Trojan Detection Method with Double-Sided Features

Using a Recurrent Neural Network and Restricted Boltzmann Machines for Malicious Traffic Detection

MBTree: Detecting Encryption RATs Communication Using Malicious Behavior Tree

A method based on hierarchical spatiotemporal features for trojan traffic detection

Uncovering APT Malware Traffic Using Deep Learning Combined with Time Sequence and Association Analysis

Network Traffic Anomaly Detection Using Recurrent Neural Networks

An Unknown Trojan Detection Method Based on Software Network Behavior

HSTF-Model: An HTTP-based Trojan detection model via the Hierarchical Spatio-temporal Features of Traffics

A Novel Malware Encrypted Traffic Detection Framework Based on Ensemble Learning

An Encrypted Malicious Traffic Detection System Based on Neural Network

BotMark: Automated botnet detection with hybrid analysis of flow-based and graph-based traffic behaviors

An Intelligent Network Attack Detection Method Based on RNN

An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks

Network Anomaly Intrusion Detection Based on Deep Learning Approach

RT-Droid: a novel approach for real-time android application analysis with transfer learning-based CNN models

Effective detection of mobile malware behavior based on explainable deep neural network