Abstract:The HPC community is actively researching and evaluating tools to support execution of scientific applications in cloud-based environments. Among the various technologies, containers have recently gained importance as they have significantly better performance compared to full-scale virtualization, support for microservices and DevOps, and work seamlessly with workflow and orchestration tools. Docker is currently the leader in containerization technology because it offers low overhead, flexibility, portability of applications, and reproducibility. Singularity is another container solution that is of interest as it is designed specifically for scientific applications. It is important to conduct performance and feature analysis of the container technologies to understand their applicability for each application and target execution environment. This paper presents a (1) performance evaluation of Docker and Singularity on bare metal nodes in the Chameleon cloud (2) mechanism by which Docker containers can be mapped with InfiniBand hardware with RDMA communication and (3) analysis of mapping elements of parallel workloads to the containers for optimal resource management with container-ready orchestration tools. Our experiments are targeted toward application developers so that they can make informed decisions on choosing the container technologies and approaches that are suitable for their HPC workloads on cloud infrastructure. Our performance analysis shows that scientific workloads for both Docker and Singularity based containers can achieve near-native performance. Singularity is designed specifically for HPC workloads. However, Docker still has advantages over Singularity for use in clouds as it provides overlay networking and an intuitive way to run MPI applications with one container per rank for fine-grained resources allocation.

Container solutions for HPC Systems: A Case Study of Using Shifter on Blue Waters

Houdini's Escape

Containers for portable, productive and performant scientific computing

Container orchestration on HPC systems through Kubernetes

Survey of adaptive containerization architectures for HPC

Power Containers

Portability and Scalability Evaluation of Large-Scale Statistical Modeling and Prediction Software through HPC-Ready Containers

Containerisation for High Performance Computing Systems: Survey and Prospects

Container Orchestration on HPC Systems

Evaluating integration and performance of containerized climate applications on a Hewlett Packard Enterprise Cray system

Performant Container Support for HPC Applications

Self-Scaling Clusters and Reproducible Containers to Enable Scientific Computing

Kub: Enabling Elastic HPC Workloads on Containerized Environments

Implementation and evaluation of a container management platform on Docker: Hadoop deployment as an example

HPC with Enhanced User Separation

Reproducible and Portable Workflows for Scientific Computing and HPC in the Cloud

Evaluation of Docker Containers for Scientific Workloads in the Cloud

Cost-efficient Workflow as a Service using Containers

A containerized analytics framework for data and compute-intensive pipeline applications

A Parallel Computing Model for Container Terminal Logistics

HPX with Spack and Singularity Containers: Evaluating Overheads for HPX/Kokkos using an astrophysics application