Abstract:Since the beginning of the 21st century, the use of cloud computing has increased rapidly, and it currently plays a significant role among most organizations' information technology (IT) infrastructure. Virtualization technologies, particularly virtual machines (VMs), are widely used and lie at the core of cloud computing. While different operating systems can run on top of VM instances, in public cloud environments the Linux operating system is used 90% of the time. Because of their prevalence, organizational Linux-based virtual servers have become an attractive target for cyber-attacks, mainly launched by sophisticated malware designed at causing harm, sabotaging operations, obtaining data, or gaining financial profit. This has resulted in the need for an advanced and reliable unknown malware detection mechanism for Linux cloud-based environments. Antivirus software and today's even more advanced malware detection solutions have limitations in detecting new, unseen, and evasive malware. Moreover, many existing solutions are considered untrusted, as they operate on the inspected machine and can be interfered with, and can even be detected by the malware itself, allowing malware to evade detection and cause damage. In this paper, we propose Deep-Hook, a trusted framework for unknown malware detection in Linux-based cloud environments. Deep-Hook hooks the VM's volatile memory in a trusted manner and acquires the memory dump to discover malware footprints while the VM operates. The memory dumps are transformed into visual images which are analyzed using a convolutional neural network (CNN) based classifier. The proposed framework has some key advantages, such as its agility, its ability to eliminate the need for features defined by a cyber domain expert, and most importantly, its ability to analyze the entire memory dump and thus to better utilize the existing indication it conceals, thus allowing the induction of a more accurate detection model. Deep-Hook was evaluated on widely used Linux virtual servers; four state-of-the-art CNN architectures; eight image resolutions; and a total of 22,400 volatile memory dumps representing the execution of a broad set of benign and malicious Linux applications. Our experimental evaluation results demonstrate Deep-Hook's ability to effectively, efficiently, and accurately detect and classify unknown malware (even evasive malware like rootkits), with an AUC and accuracy of up to 99.9%.

DockerWatch: a two-phase hybrid detection of malware using various static features in container cloud

A malware detection framework based on kolmogorov complexity

KubAnomaly: Anomaly detection for the Docker orchestration platform with neural network approaches

MDCD: A malware detection approach in cloud using deep learning

Malware Detection Based on Multi-level and Dynamic Multi-feature Using Ensemble Learning at Hypervisor

ContainerGuard: A Real-Time Attack Detection System in Container-Based Big Data Platform

Whispers Between The Containers: High-Capacity Covert Channel Attacks In Docker

Intrusion Detection System for Applications using Linux Containers

Deep-Hook: A trusted deep learning-based framework for unknown malware detection and classification in Linux cloud environments

An Android Malware Detection Method Using Deep Learning Based on API Calls

A Malware Detection Method Based on Sandbox, Binary Instrumentation and Multidimensional Feature Extraction

A Hybrid Deep Network Framework for Android Malware Detection

Experimental Analysis of Security Attacks for Docker Container Communications

Enhancing Linux System Security: A Kernel-Based Approach to Fileless Malware Detection and Mitigation

Accurate mobile malware detection and classification in the cloud

Leveraging malicious behavior traces from volatile memory using machine learning methods for trusted unknown malware detection in Linux cloud environments

A Novel Approach to Malicious Code Detection Using CNN-BiLSTM and Feature Fusion

A hybrid deep learning image-based analysis for effective malware detection

Longitudinal Risk-based Security Assessment of Docker Software Container Images

Ab‐HIDS: An anomaly‐based host intrusion detection system using frequency of N‐gram system call features and ensemble learning for containerized environment

Deep hybrid approach with sequential feature extraction and classification for robust malware detection