ADSAD: An unsupervised attention-based discrete sequence anomaly detection framework for network security analysis

Zhi-Quan Qin,Xing-Kong Ma,Yong-Jun Wang
DOI: https://doi.org/10.1016/j.cose.2020.102070
2020-12-01
Abstract:<p>Detecting anomalous discrete sequences such as payloads and syscall traces is a crucial task of network security analysis for discovering novel attacks. The data characteristics that lack of labels, very long sequences and irregularly variable lengths make generating proper representations for the sequences for anomaly detection quite challenging. Traditional methods combining shallow models with feature engineering require lots of time and effort from researchers. And they only catch short patterns for the sequences. Recently deep learning is paid more and more attention due to its excellent performance on data representation. Current works simply adopt recurrent neural network based models to this task. They learn the local patterns of the sequences but can not view the sequences globally. Besides, the variable length makes the deep models that accept fixed-size inputs unavailable. Moreover, the deep models usually lack interpretability. Here an unsupervised deep learning framework utilizing attention mechanism called ADSAD is proposed to address these issues. ADSAD takes both the data characteristics and the limitation of the deep models into consideration and generate the global representations for the sequences by two steps, in which the attention mechanism is applied to improve the interpretability. The empirical results showed that the ADSAD instances significantly outperformed the state-of-the-art deep models, with the relative AUC improvement of up to 7%. The attention mechanism not only enhanced the detection performance by up to 73% in terms of AUC but was also able to assist experts for anomaly analysis by visualization.</p>
computer science, information systems
What problem does this paper attempt to address?