Abstract:The HPC community is actively researching and evaluating tools to support execution of scientific applications in cloud-based environments. Among the various technologies, containers have recently gained importance as they have significantly better performance compared to full-scale virtualization, support for microservices and DevOps, and work seamlessly with workflow and orchestration tools. Docker is currently the leader in containerization technology because it offers low overhead, flexibility, portability of applications, and reproducibility. Singularity is another container solution that is of interest as it is designed specifically for scientific applications. It is important to conduct performance and feature analysis of the container technologies to understand their applicability for each application and target execution environment. This paper presents a (1) performance evaluation of Docker and Singularity on bare metal nodes in the Chameleon cloud (2) mechanism by which Docker containers can be mapped with InfiniBand hardware with RDMA communication and (3) analysis of mapping elements of parallel workloads to the containers for optimal resource management with container-ready orchestration tools. Our experiments are targeted toward application developers so that they can make informed decisions on choosing the container technologies and approaches that are suitable for their HPC workloads on cloud infrastructure. Our performance analysis shows that scientific workloads for both Docker and Singularity based containers can achieve near-native performance. Singularity is designed specifically for HPC workloads. However, Docker still has advantages over Singularity for use in clouds as it provides overlay networking and an intuitive way to run MPI applications with one container per rank for fine-grained resources allocation.

Evaluating integration and performance of containerized climate applications on a Hewlett Packard Enterprise Cray system

Containers for portable, productive and performant scientific computing

Portability and Scalability Evaluation of Large-Scale Statistical Modeling and Prediction Software through HPC-Ready Containers

HPX with Spack and Singularity Containers: Evaluating Overheads for HPX/Kokkos using an astrophysics application

Performance evaluation of container-based virtualization for high performance computing environments

Evaluation of Docker Containers for Scientific Workloads in the Cloud

Understanding Layered Portability from HPC to Cloud in Containerized Environments

Survey of adaptive containerization architectures for HPC

Container solutions for HPC Systems: A Case Study of Using Shifter on Blue Waters

Reproducible and Portable Workflows for Scientific Computing and HPC in the Cloud

A containerized analytics framework for data and compute-intensive pipeline applications

A directive based hybrid met office NERC cloud model

Containerisation for High Performance Computing Systems: Survey and Prospects

Kub: Enabling Elastic HPC Workloads on Containerized Environments

Machine Learning Algorithms for Active Monitoring of High Performance Computing as a Service (HPCaaS) Cloud Environments

Solving global shallow water equations on heterogeneous supercomputers

Early experiences evaluating the HPE/Cray ecosystem for AMD GPUs

Performance on HPC Platforms Is Possible Without C++

ContainerStress: Autonomous Cloud-Node Scoping Framework for Big-Data ML Use Cases

Container orchestration on HPC systems through Kubernetes

Carbon Containers: A System-level Facility for Managing Application-level Carbon Emissions