Abstract:Nowadays, the rapid growth of the number and variety of malware brings great security challenges. Machine learning has become a mainstream tool for effective malware detection, which can mainly be classified into static and dynamic analysis methods. The purpose of malware detection is to have a good and stable detection performance for different software forms. However, many static analysis methods are easily affected by packing and other code obfuscation techniques, and dynamic analysis methods are commonly believed more robust, while the impact of packing on them has received little attention. In addition, adversarial sample attacks against dynamic analysis methods have also been conducted. This indicates the need to investigate more accurate and robust malware classification methods. In this paper, we propose a new robust dynamic analysis method for malware detection by using specific fine-grained behavioral features, i.e., control flow traces. Further, a malware classifier is constructed by converting control flow traces into byte sequences and applying a combination of convolutional neural networks and long short term memory . The proposed classifier can effectively detect malware with an accuracy of up to 95.7%, as well as detect unseen malware with an accuracy of 94.6% (indicating a good performance in handling the evolution of malware). Meanwhile, it is first found experimentally that packing has specific interference with existing behavior-based malware classifiers, resulting in even worse performance than static classifiers in some cases. However, the proposed classifier performs well in terms of robustness, showing stable performance under the interference of an uneven packing distribution in the dataset, with a 26.3% higher true positive rate for unpacked samples compared to the API call-based classifier. In addition, the classifier is also more robust against adversarial samples, with a detection rate of at least 83%.

A fine-grained classification approach for the packed malicious code

PackerGrind: An Adaptive Unpacking System for Android Apps.

Happer - Unpacking Android Apps via a Hardware-Assisted Approach.

Parema: an Unpacking Framework for Demystifying VM-based Android Packers.

Analysis of machine learning approaches to packing detection

An Efficient Multi-Step Framework for Malware Packing Identification

Structural Feature Based Anomaly Detection for Packed Executable Identification

FMDiv: Functional Module Division on Binary Malware for Accurate Malicious Code Localization.

Packer Identification Using Hidden Markov Model

Assessing the Impact of Packing on Machine Learning-Based Malware Detection and Classification Systems

Efficient and Robust Malware Detection Based on Control Flow Traces Using Deep Neural Networks

MLxPack: Investigating the Effects of Packers on ML-based Malware Detection Systems Using Static and Dynamic Traits

Precise system-wide concatic malware unpacking

Malicious Code Detection Method Based on Static Features and Ensemble Learning

A malware detection framework based on kolmogorov complexity

K-Means Clustering Analysis Based On Adaptive Weights For Malicious Code Detection

Learning-Based Detection for Malicious Android Application Using Code Vectorization

Efficient Detection and Classification of Internet-of-Things Malware Based on Byte Sequences from Executable Files

2-SPIFF: a 2-stage packer identification method based on function call graph and file attributes

Component Similarity Based Methods for Automatic Analysis of Malicious Executables