Migration of CMSWEB Cluster at CERN to Kubernetes

Muhammad Imran,Valentin Kuznetsov,Lina Marcella,Katarzyna Maria Dziedziniewicz-Wojcik,Andreas Pfeiffer,Panos Paparrigopoulos
DOI: https://doi.org/10.22323/1.390.0911
2021-02-25
Abstract:The CMS experiment heavily relies on the CMSWEB cluster to host critical services for its operational needs. The cluster is deployed on virtual machines (VMs) from the CERN OpenStack cloud and is manually maintained by operators and developers. The release cycle is composed of several steps, from building RPMs, their deployment to perform validation, and integration tests. To enhance the sustainability of the CMSWEB cluster, CMS decided to migrate its cluster to a containerized solution such as Docker, orchestrated with Kubernetes (k8s). This allows us to significantly reduce the release upgrade cycle, follow the end-to-end deployment procedure, and reduce operational cost. This paper gives an overview of the current CMSWEB cluster and its issues. We describe the new architecture of the CMSWEB cluster in Kubernetes. We also provide a comparison of VM and Kubernetes deployment approaches and report on lessons learned during the migration process.
Distributed, Parallel, and Cluster Computing
What problem does this paper attempt to address?