Abstract:The Remote Access Trojan (RAT), whose exposure often lags far behind its widespread infection, plays a part in the growing number of cyber-attacks. In terms of intrusion detection, signature-based methods still occupy the dominant position together with anomaly-based methods that are deployed to be complementary. The anomaly-based methods are efficient and resource saving, however, anomaly-based RAT detection mainly utilizes machine learning on collective features without considering packet sequences. In addition, imbalanced data has a negative impact on machine learning classification. Accordingly, we analyzed packet sequences from various applications and found a general correlation between the packet direction sequences and the external control behaviors of RATs. This research presents a novel framework for tracking external control at the borders of area networks. We extract packet payload-size sequences, inter-arrival time sequences and packet direction sequences of IP flows. Then, RAT network behavior is exposed in the packet direction sequences of flow slices, which are generated by the inter-arrival time sequences. Naïve Bayes is utilized for classification, and a frequent sequence mining algorithm is implemented to eliminate noise. After deploying the device, we detected all the RAT sample sessions. Our method achieves a false positive rate of less than 0.6% on real-world campus network data, which demonstrates its efficacy.

Detecting Remote Access Trojans Through External Control at Area Network Borders