Abstract:Malware is one of the most significant threats in today’s computing world since the number of websites distributing malware is increasing at a rapid rate. Malware analysis and prevention methods are increasingly becoming necessary for computer systems connected to the Internet. This software exploits the system’s vulnerabilities to steal valuable information without the user’s knowledge, and stealthily send it to remote servers controlled by attackers. Traditionally, anti-malware products use signatures for detecting known malware. However, the signature-based method does not scale in detecting obfuscated and packed malware. Considering that the cause of a problem is often best understood by studying the structural aspects of a program like the mnemonics, instruction opcode, API Call, etc. In this paper, we investigate the relevance of the features of unpacked malicious and benign executables like mnemonics, instruction opcodes, and API to identify a feature that classifies the executable. Prominent features are extracted using Minimum Redundancy and Maximum Relevance (mRMR) and Analysis of Variance (ANOVA). Experiments were conducted on four datasets using machine learning and deep learning approaches such as Support Vector Machine (SVM), Naïve Bayes, J48, Random Forest (RF), and XGBoost. In addition, we also evaluate the performance of the collection of deep neural networks like Deep Dense network, One-Dimensional Convolutional Neural Network (1D-CNN), and CNN-LSTM in classifying unknown samples, and we observed promising results using APIs and system calls. On combining APIs/system calls with static features, a marginal performance improvement was attained comparing models trained only on dynamic features. Moreover, to improve accuracy, we implemented our solution using distinct deep learning methods and demonstrated a fine-tuned deep neural network that resulted in an F1-score of 99.1% and 98.48% on Dataset-2 and Dataset-3, respectively.

Detecting Malignant TLS Servers Using Machine Learning Techniques

Deciphering Malware's use of TLS (without Decryption)

Unsupervised Detection and Clustering of Malicious TLS Flows

SSL Malicious Traffic Detection Based On Multi-view Features.

A Novel Framework for Learning to Detect Malicious Web Pages

Detecting Malicious TLS Network Traffic Based on Communication Channel Features

Detection of Malicious Software by Analyzing Distinct Artifacts Using Machine Learning and Deep Learning Algorithms

Significance of Machine Learning for Detection of Malicious Websites on an Unbalanced Dataset

Malware Analysis Using Machine Learning and Deep Learning Techniques

Malicious URL Detection using Machine Learning: A Survey

Two-layer detection framework with a high accuracy and efficiency for a malware family over the TLS protocol

Identifying Malicious Web Domains Using Machine Learning Techniques with Online Credibility and Performance Data.

Malware Analysis and Detection Using Machine Learning Algorithms

Detection of Malicious Websites Using Machine Learning Techniques

TLS fingerprint for encrypted malicious traffic detection with attributed graph kernel

Malware Detection Module using Machine Learning Algorithms to Assist in Centralized Security in Enterprise Networks

A novel TLS-based Fingerprinting approach that combines feature expansion and similarity mapping

Real-time Detection of Malicious Web Pages Based on Statistical Learning

Machine Learning System for Malicious Website Detection: A Literature Review

Malicious Threats Detection of Executable File

Machine Learning Based Malicious Node Detection in IoT Environment