On Evaluating Multi-class Network Traffic Classifiers Based on AUC

Jie Yang,Yi-Xuan Wang,Yuan-Yuan Qiao,Xiao-Xing Zhao,Fang Liu,Gang Cheng
DOI: https://doi.org/10.1007/s11277-015-2473-4
IF: 2.017
2015-01-01
Wireless Personal Communications
Abstract:Traffic monitoring and traffic characterization are essential for network planning and operation. Machine learning has been readily applied to high-speed network traffic classification. Evaluating the viability and performance of various classifiers for different application scenarios, particularly for Internet traffic, is critical. Based on our research, commonly used metrics (such as accuracy) can’t accurately reflect the classifier performance in imbalanced data sets. Other methods, like ROC curve, or simply AUC can’t perform well in multi-objective classification. Network traffic is an imbalanced multi-class data set. To the best of our knowledge, no research has been conducted to quantitatively compare performance in the network traffic classification. To address these issues, we propose an evaluation method aiming for the case of imbalanced multi-class network traffic classification. This proposal is based on the multi-objective metric, an area under the ROC (receiver operating characteristic) curve, or simply AUC (the area under the ROC). We conduct our experiments with the traffic trace captured from real network. The experiment results show that our method is capable of evaluating the case of the classes being misclassified. Particularly, it is more sensitive for the case of small proportion (Minority classes) being misclassified into large proportion (Majority classes) than the case of Majority classes being misclassified into Minority classes. Hence, we recommend to leverage the measure proposed in this paper to evaluate the classifier performance in distinguishing of misclassifying the Minority applications into the Majority applications.
What problem does this paper attempt to address?