Abstract:Malware is one of the most significant threats in today’s computing world since the number of websites distributing malware is increasing at a rapid rate. Malware analysis and prevention methods are increasingly becoming necessary for computer systems connected to the Internet. This software exploits the system’s vulnerabilities to steal valuable information without the user’s knowledge, and stealthily send it to remote servers controlled by attackers. Traditionally, anti-malware products use signatures for detecting known malware. However, the signature-based method does not scale in detecting obfuscated and packed malware. Considering that the cause of a problem is often best understood by studying the structural aspects of a program like the mnemonics, instruction opcode, API Call, etc. In this paper, we investigate the relevance of the features of unpacked malicious and benign executables like mnemonics, instruction opcodes, and API to identify a feature that classifies the executable. Prominent features are extracted using Minimum Redundancy and Maximum Relevance (mRMR) and Analysis of Variance (ANOVA). Experiments were conducted on four datasets using machine learning and deep learning approaches such as Support Vector Machine (SVM), Naïve Bayes, J48, Random Forest (RF), and XGBoost. In addition, we also evaluate the performance of the collection of deep neural networks like Deep Dense network, One-Dimensional Convolutional Neural Network (1D-CNN), and CNN-LSTM in classifying unknown samples, and we observed promising results using APIs and system calls. On combining APIs/system calls with static features, a marginal performance improvement was attained comparing models trained only on dynamic features. Moreover, to improve accuracy, we implemented our solution using distinct deep learning methods and demonstrated a fine-tuned deep neural network that resulted in an F1-score of 99.1% and 98.48% on Dataset-2 and Dataset-3, respectively.

Efficient Malware Packer Identification Using Support Vector Machines with Spectrum Kernel

An Efficient Multi-Step Framework for Malware Packing Identification

Packer Identification Using Hidden Markov Model

A fine-grained classification approach for the packed malicious code

Assessing the Impact of Packing on Machine Learning-Based Malware Detection and Classification Systems

MLxPack: Investigating the Effects of Packers on ML-based Malware Detection Systems Using Static and Dynamic Traits

PackerGrind: An Adaptive Unpacking System for Android Apps.

Analysis of machine learning approaches to packing detection

Happer - Unpacking Android Apps via a Hardware-Assisted Approach.

Parema: an Unpacking Framework for Demystifying VM-based Android Packers.

SpecView: Malware Spectrum Visualization Framework With Singular Spectrum Transformation

Structural Feature Based Anomaly Detection for Packed Executable Identification

Precise system-wide concatic malware unpacking

2-SPIFF: a 2-stage packer identification method based on function call graph and file attributes

A malware detection framework based on kolmogorov complexity

Malware Analysis Using Machine Learning and Deep Learning Techniques

Detection of Malicious Software by Analyzing Distinct Artifacts Using Machine Learning and Deep Learning Algorithms

Efficient Detection and Classification of Internet-of-Things Malware Based on Byte Sequences from Executable Files

Smart OMVI: Obfuscated Malware Variant Identification using a novel dataset

Efficient and Robust Malware Detection Based on Control Flow Traces Using Deep Neural Networks