Abstract:Conventional malware detection approaches have the overhead of feature extraction, the requirement of domain experts, and are time-consuming and resource-intensive. Learning-based approaches are the mainstay of malware detection as they overcome most of these challenges by significantly improving the detection effectiveness and providing a low false positive rate. The exponential growth of malware variants and first-time-appeared malware, which includes polymorphic and zero-day attacks, are some of the significant challenges to learning-based malware detectors. These challenges have catastrophic impacts on the detection effectiveness of these learning-based malware detectors. This paper proposes a novel deep learning-based framework to detect first-time-appeared malware effectively and efficiently by providing better performance than conventional malware detection approaches. First, it translates and visualises each Windows portable executable (PE) file into a coloured image to eliminate the overhead of feature extraction and the need for domain experts to analyse the features. In the subsequent step, a fine-tuned deep learning model is used to extract the deep features from the last fully connected layer. The step has reduced the cost of training required by the deep learning models if used for end-to-end classification. The third step selects the most important and influential features through a powerful feature selection algorithm. The most important features are then fed to a one-class classifier for final detection. With the one-class classifier, an enclosed boundary around the features of benign data is constructed. Anything outside the boundary is declared as an anomaly/malicious. It has enhanced the framework's ability to detect evolving, unseen, polymorphic, and zero-day attacks, as well as reducing the problem of overfitting. The detection effectiveness of the proposed framework is validated with state-of-the-art deep learning models and conventional approaches. The proposed framework has outperformed with an accuracy of 99.30% on the Malimg dataset. The Wilcoxon signed-rank test is used to validate the statistical significance of the proposed framework. It is evident from the results that the proposed framework is effective and can be used in the defence industry, resulting in more powerful and robust solutions against zero-day and polymorphic attacks.

Deep Learning for Zero-Day Malware Detection and Classification: A Survey

A Survey of the Recent Trends in Deep Learning Based Malware Detection

Malware Analysis Using Machine Learning and Deep Learning Techniques

A Survey of Malware Detection Using Deep Learning

Deep Learning Based Hybrid Analysis of Malware Detection and Classification: A Recent Review

Robust Intelligent Malware Detection Using Deep Learning

The rise of machine learning for detection and classification of malware: Research developments, trends and challenges

Deep Learning Models for Detecting Malware Attacks

Deep learning-powered malware detection in cyberspace: a contemporary review

Deep Learning Methods for Malware and Intrusion Detection: A Systematic Literature Review

From Zero-Shot Machine Learning to Zero-Day Attack Detection

A Review of Deep Learning Based Malware Detection Techniques

A Survey on Machine Learning-based Detection and Classification Technology of Malware

A Malware Classification Survey on Adversarial Attacks and Defences

Artificial Intelligence-Based Malware Detection, Analysis, and Mitigation

A novel machine learning approach for detecting first-time-appeared malware

Efficient Malware Detection Using Hybrid Approach of Transfer Learning and Generative Adversarial Examples with Image Representation

An investigation of a deep learning based malware detection system

ZeVigilante: Detecting Zero-Day Malware Using Machine Learning and Sandboxing Analysis Techniques

Zero-day attack detection: a systematic literature review

Deep Neural Network Based Malware Detection Using Two Dimensional Binary Program Features