Abstract:Malware is one of the most significant threats in today’s computing world since the number of websites distributing malware is increasing at a rapid rate. Malware analysis and prevention methods are increasingly becoming necessary for computer systems connected to the Internet. This software exploits the system’s vulnerabilities to steal valuable information without the user’s knowledge, and stealthily send it to remote servers controlled by attackers. Traditionally, anti-malware products use signatures for detecting known malware. However, the signature-based method does not scale in detecting obfuscated and packed malware. Considering that the cause of a problem is often best understood by studying the structural aspects of a program like the mnemonics, instruction opcode, API Call, etc. In this paper, we investigate the relevance of the features of unpacked malicious and benign executables like mnemonics, instruction opcodes, and API to identify a feature that classifies the executable. Prominent features are extracted using Minimum Redundancy and Maximum Relevance (mRMR) and Analysis of Variance (ANOVA). Experiments were conducted on four datasets using machine learning and deep learning approaches such as Support Vector Machine (SVM), Naïve Bayes, J48, Random Forest (RF), and XGBoost. In addition, we also evaluate the performance of the collection of deep neural networks like Deep Dense network, One-Dimensional Convolutional Neural Network (1D-CNN), and CNN-LSTM in classifying unknown samples, and we observed promising results using APIs and system calls. On combining APIs/system calls with static features, a marginal performance improvement was attained comparing models trained only on dynamic features. Moreover, to improve accuracy, we implemented our solution using distinct deep learning methods and demonstrated a fine-tuned deep neural network that resulted in an F1-score of 99.1% and 98.48% on Dataset-2 and Dataset-3, respectively.

Deciphering Malware's use of TLS (without Decryption)

Unsupervised Detection and Clustering of Malicious TLS Flows

Detecting Malignant TLS Servers Using Machine Learning Techniques

Malware Analysis Using Machine Learning and Deep Learning Techniques

Feature Analysis of Encrypted Malicious Traffic

A Novel Framework for Malicious Encrypted Traffic Classification at Host Level and Flow Level.

A Novel Malware Encrypted Traffic Detection Framework Based on Ensemble Learning

Encrypted Malicious Traffic Detection Based on Word2Vec

TLS fingerprint for encrypted malicious traffic detection with attributed graph kernel

Catch'em all: Classification of Rare, Prominent, and Novel Malware Families

A Longitudinal Study of Cryptographic API: a Decade of Android Malware

MalPhase: Fine-Grained Malware Detection Using Network Flow Data

Malware Detection and Analysis Tools

Malware Analysis and Detection Using Machine Learning Algorithms

Detection of Malicious Software by Analyzing Distinct Artifacts Using Machine Learning and Deep Learning Algorithms

MalwareDNA: Simultaneous Classification of Malware, Malware Families, and Novel Malware

Decoding the Secrets of Machine Learning in Malware Classification: A Deep Dive into Datasets, Feature Extraction, and Model Performance

Tools and Techniques for Malware Detection and Analysis

Classifying Tor Traffic Encrypted Payload Using Machine Learning

Feature-level Malware Obfuscation in Deep Learning

DeepMAL -- Deep Learning Models for Malware Traffic Detection and Classification