Abstract:Cloud computing has become a compelling paradigm built on compute and storage virtualization technologies. The current virtualization solution in the Cloud widely relies on hypervisor-based technologies. Given the recent booming of the container ecosystem, the container-based virtualization starts receiving more attention for being a promising alternative. Although the container technologies are generally considered to be lightweight, no virtualization solution is ideally resource-free, and the corresponding performance overheads will lead to negative impacts on the quality of Cloud services. To facilitate understanding container technologies from the performance engineering's perspective, we conducted two-stage performance investigations into Docker containers as a concrete example. At the first stage, we used a physical machine with “just-enough” resource as a baseline to investigate the performance overhead of a standalone Docker container against a standalone virtual machine (VM). With findings contrary to the related work, our evaluation results show that the virtualization's performance overhead could vary not only on a feature-by-feature basis but also on a job-to-job basis. Moreover, the hypervisor-based technology does not come with higher performance overhead in every case. For example, Docker containers particularly exhibit lower QoS in terms of storage transaction speed. At the ongoing second stage, we employed a physical machine with “fair-enough” resource to implement a container-based MapReduce application and try to optimize its performance. In fact, this machine failed in affording VM-based MapReduce clusters in the same scale. The performance tuning results show that the effects of different optimization strategies could largely be related to the data characteristics. For example, LZO compression can bring the most significant performance improvement when dealing with text data in our case. (Less)

Performant Container Support for HPC Applications

Evaluation of Docker Containers for Scientific Workloads in the Cloud

Self-Scaling Clusters and Reproducible Containers to Enable Scientific Computing

Singularity: Scientific containers for mobility of compute

Houdini's Escape

Containers for portable, productive and performant scientific computing

Performance evaluation of container-based virtualization for high performance computing environments

Container orchestration on HPC systems through Kubernetes

Containerisation for High Performance Computing Systems: Survey and Prospects

Evaluating integration and performance of containerized climate applications on a Hewlett Packard Enterprise Cray system

HPX with Spack and Singularity Containers: Evaluating Overheads for HPX/Kokkos using an astrophysics application

Two-Stage Performance Engineering of Container-based Virtualization

Survey of adaptive containerization architectures for HPC

Container solutions for HPC Systems: A Case Study of Using Shifter on Blue Waters

Container Orchestration on HPC Systems

H-Container: Enabling Heterogeneous-ISA Container Migration in Edge Computing

Understanding Layered Portability from HPC to Cloud in Containerized Environments

Fine-Grained Scheduling for Containerized HPC Workloads in Kubernetes Clusters

Performance on HPC Platforms Is Possible Without C++

Reproducible and Portable Workflows for Scientific Computing and HPC in the Cloud