Docker: lightweight Linux containers for consistent development and deployment

D. Merkel

2014-03-01

Abstract:Docker promises the ability to package applications and their dependencies into lightweight containers that move easily between different distros, start up quickly and are isolated from each other.

Engineering,Computer Science

What problem does this paper attempt to address?

Houdini's Escape

Xing Gao,Zhongshu Gu,Zhengfa Li,Hani Jamjoom,Cong Wang

DOI: https://doi.org/10.1145/3319535.3354227

2019-01-01

Abstract:Linux Control Groups, i.e., cgroups, are the key building blocks to enable operating-system-level containerization. The cgroups mechanism partitions processes into hierarchical groups and applies different controllers to manage system resources, including CPU, memory, block I/O, etc. Newly spawned child processes automatically copy cgroups attributes from their parents to enforce resource control. Unfortunately, inherited cgroups confinement via process creation does not always guarantee consistent and fair resource accounting. In this paper, we devise a set of exploiting strategies to generate out-of-band>workloads via de-associating processes from their original process groups. The system resources consumed by such workloads will not be charged to the appropriate cgroups. To further demonstrate the feasibility, we present five case studies within Docker containers to demonstrate how to break the resource rein of cgroups in realistic scenarios. Even worse, by exploiting those cgroups' insufficiencies in a multi-tenant container environment, an adversarial container is able to greatly amplify the amount of consumed resources, significantly slow-down other containers on the same host, and gain extra unfair advantages on the system resources. We conduct extensive experiments on both a local testbed and an Amazon EC2 cloud dedicated server. The experimental results demonstrate that a container can consume system resources (e.g., CPU) as much as $200\times$ of its limit, and reduce both computing and I/O performance of particular workloads in other co-resident containers by 95%.
Software Application Development Using Container Technology

Dražen Marinković,Velimir Kojić,Zoran Ž. Avramović

DOI: https://doi.org/10.7251/jit2101054m

2021-09-20

Abstract:The paper will give an example and an overview of how we can set up and maintain web applications using a Docker. We will define what Docker is, what containers are and how to use them. Developers often find themselves in a situation where their program works properly on a computer in the laboratory environment in which the application was developed, but after installing the program on the production server, the program does not work as expected. In such circumstances, it is difficult for a programmer to determine why a program is not working. Docker solves this problem by placing applications and virtual containers that run on the same operating system.
An empirical analysis of the Docker container ecosystem on GitHub

Jürgen Cito,Gerald Schermann,Erik Wittern,Philipp Leitner,Sali Zumberi,Harald C. Gall

DOI: https://doi.org/10.7287/peerj.preprints.2905v1

2017-04-03

Abstract:Docker allows packaging an application with its dependencies into a standardized, self-contained unit (a so-called container), which can be used for software development and to run the application on any system. Dockerfiles are declarative definitions of an environment that aim to enable reproducible builds of the container. They can often be found in source code repositories and enable the hosted software to come to life in its execution environment. We conduct an exploratory empirical study with the goal of characterizing the Docker ecosystem, prevalent quality issues, and the evolution of Dockerfiles. We base our study on a data set of over 70000 Dockerfiles, and contrast this general population with samplings that contain the Top-100 and Top-1000 most popular Docker-using projects. We find that most quality issues (28.6%) arise from missing version pinning (i.e., specifying a concrete version for dependencies). Further, we were not able to build 34% of Dockerfiles from a representative sample of 560 projects. Integrating quality checks, e.g., to issue version pinning warnings, into the container build process could result into more reproducible builds. The most popular projects change more often than the rest of the Docker population, with 5.81 revisions per year and 5 lines of code changed on average. Most changes deal with dependencies, that are currently stored in a rather unstructured manner. We propose to introduce an abstraction that, for instance, could deal with the intricacies of different package managers and could improve migration to more light-weight images.
Microservice Architecture Practices and Experience: a Focused Look on Docker Configuration Files

Luciano Baresi,Giovanni Quattrocchi,Damian Andrew Tamburri

DOI: https://doi.org/10.48550/arXiv.2212.03107

2022-12-07

Abstract:Cloud applications are more and more microservice-oriented, but a concrete charting of the microservices architecture landscape -- namely, the space of technical options available for microservice software architects in their decision-making -- is still very much lacking, thereby limiting the ability of software architects to properly evaluate their architectural decisions with sound experiential devices and/or practical design principles. On the one hand, Microservices are fine-grained, loosely coupled services that communicate through lightweight protocols. On the other hand, each microservice can use a different software stack, be deployed and scaled independently or even executed in different containers, which provide isolation and a wide-range of configuration options but also offer unforeseeable architectural interactions and under-explored architecture smells, with such experience captured mainly in software repositories where such solutions are cycled. This paper adopts a mining software repositories (MSR) approach to capture the practice within the microservice architecture landscape, by eliciting and analysing Docker configuration files, being Docker the leading technical device to design for, and implement modern microservices. Our analysis of Docker-based microservices gives an interesting summary of the current state of microservices practice and experience. Conversely, observing that all our datapoints have their own shape and characteristics, we conclude that further comparative assessment with industrial systems is needed to better address the recurring positive principles and patterns around microservices.

Software Engineering
Containerization and Orchestration: Implementing OpenShift and Docker

Sowmith Daram,Prof.(Dr.) Arpit Jain,Er. Om Goel

DOI: https://doi.org/10.36676/irt.v7.i4.1457

2021-12-30

Abstract:Containerization and orchestration have revolutionized the deployment and management of applications, offering unprecedented levels of efficiency, scalability, and consistency. This paper delves into the intricacies of implementing two leading technologies in this domain: Docker for containerization and OpenShift for orchestration. Docker has emerged as a foundational tool that packages applications and their dependencies into portable containers, enabling consistent execution across various environments. Meanwhile, OpenShift, a Kubernetes-based platform, extends Docker’s capabilities by providing robust tools for container orchestration, application deployment, and scaling. The paper explores the architectural components of Docker and OpenShift, emphasizing how they complement each other to form a comprehensive solution for modern application management. It highlights the benefits of using Docker containers, such as lightweight execution, isolation, and ease of integration into CI/CD pipelines. Moreover, it discusses OpenShift’s orchestration capabilities, including automated scaling, load balancing, and self-healing, which are essential for maintaining high availability and performance in dynamic environments.
A qualitative and quantitative analysis of container engines

Luciano Baresi,Giovanni Quattrocchi,Nicholas Rasi

DOI: https://doi.org/10.1016/j.jss.2024.111965

IF: 3.5

2024-01-07

Journal of Systems and Software

Abstract:Containerization is a virtualization technique that allows one to create and run executables consistently on any infrastructure. Compared to virtual machines, containers are lighter since they do not bundle a (guest) operating system but they share its kernel, and they only include the files, libraries, and dependencies that are required to properly execute a process. In the past few years, multiple container engines (i.e., tools for configuring, executing, and managing containers) have been developed ranging from some that are "general purpose", and mostly employed for Cloud executions, to others that are built for specific contexts, namely Internet of Things and High-Performance Computing. Given the importance of this technology for many practitioners and researchers, this paper analyses six state-of-the-art container engines and compares them through a comprehensive study of their characteristics and performance. The results are organized around 0 findings that aim to help the readers understand the differences among the technologies and help them choose the best approach for their needs. Editor's note: Open Science material was validated by the Journal of Systems and Software Open Science Board .

computer science, theory & methods, software engineering
Containers for portable, productive and performant scientific computing

Jack S. Hale,Lizao Li,Chris N. Richardson,Garth N. Wells

DOI: https://doi.org/10.1109/MCSE.2017.2421459

2016-11-04

Abstract:Containers are an emerging technology that hold promise for improving productivity and code portability in scientific computing. We examine Linux container technology for the distribution of a non-trivial scientific computing software stack and its execution on a spectrum of platforms from laptop computers through to high performance computing (HPC) systems. We show on a workstation and a leadership-class HPC system that when deployed appropriately there are no performance penalties running scientific programs inside containers. For Python code run on large parallel computers, the run time is reduced inside a container due to faster library imports. The software distribution approach and data that we present will help developers and users decide on whether container technology is appropriate for them. We also provide guidance for the vendors of HPC systems that rely on proprietary libraries for performance on what they can do to make containers work seamlessly and without performance penalty.

Distributed, Parallel, and Cluster Computing,Mathematical Software,Software Engineering
The Rockerverse: Packages and Applications for Containerization with R

Daniel Nüst,Dirk Eddelbuettel,Dom Bennett,Robrecht Cannoodt,Dav Clark,Gergely Daroczi,Mark Edmondson,Colin Fay,Ellis Hughes,Lars Kjeldgaard,Sean Lopp,Ben Marwick,Heather Nolis,Jacqueline Nolis,Hong Ooi,Karthik Ram,Noam Ross,Lori Shepherd,Péter Sólymos,Tyson Lee Swetnam,Nitesh Turaga,Charlotte Van Petegem,Jason Williams,Craig Willis,Nan Xiao

DOI: https://doi.org/10.32614/RJ-2020-007

2020-08-17

Abstract:The Rocker Project provides widely used Docker images for R across different application scenarios. This article surveys downstream projects that build upon the Rocker Project images and presents the current state of R packages for managing Docker images and controlling containers. These use cases cover diverse topics such as package development, reproducible research, collaborative work, cloud-based data processing, and production deployment of services. The variety of applications demonstrates the power of the Rocker Project specifically and containerisation in general. Across the diverse ways to use containers, we identified common themes: reproducible environments, scalability and efficiency, and portability across clouds. We conclude that the current growth and diversification of use cases is likely to continue its positive impact, but see the need for consolidating the Rockerverse ecosystem of packages, developing common practices for applications, and exploring alternative containerisation software.

Software Engineering,Distributed, Parallel, and Cluster Computing
Sustainability and Reproducibility via Containerized Computing

Robert Nagler,David Bruhwiler,Paul Moeller,Stephen Webb

DOI: https://doi.org/10.48550/arXiv.1509.08789

2015-09-29

Abstract:Recent developments in the commercial open source community have catalysed the use of Linux containers for scalable deployment of web-based applications to the cloud. Scientific software can be containerized with dependencies, configuration files, post-processing tools and even simulation results, referred to as containerized computing. This new approach promises to significantly improve sustainability, productivity and reproducibility. We present our experiences, technology, and future plans for open source containerization of software used to model particle and radiation beams. Vagrant is central to our approach, using Docker for cloud deployment and VirtualBox virtual machines for deployment to Mac OS and Windows computers. Our technology enables seamless switching between the desktop and the cloud to simplify simulation development and execution.

Software Engineering
Enabling the Deployment of Any-Scale Robotic Applications in Microservice Architectures through Automated Containerization

Jean-Pierre Busch,Lennart Reiher,Lutz Eckstein

DOI: https://doi.org/10.1109/ICRA57147.2024.10611586

2024-08-20

Abstract:In an increasingly automated world -- from warehouse robots to self-driving cars -- streamlining the development and deployment process and operations of robotic applications becomes ever more important. Automated DevOps processes and microservice architectures have already proven successful in other domains such as large-scale customer-oriented web services (e.g., Netflix). We recommend to employ similar microservice architectures for the deployment of small- to large-scale robotic applications in order to accelerate development cycles, loosen functional dependence, and improve resiliency and elasticity. In order to facilitate involved DevOps processes, we present and release a tooling suite for automating the development of microservices for robotic applications based on the Robot Operating System (ROS). Our tooling suite covers the automated minimal containerization of ROS applications, a collection of useful machine learning-enabled base container images, as well as a CLI tool for simplified interaction with container images during the development phase. Within the scope of this paper, we embed our tooling suite into the overall context of streamlined robotics deployment and compare it to alternative solutions. We release our tools as open-source software at <a class="link-external link-https" href="https://github.com/ika-rwth-aachen/dorotos" rel="external noopener nofollow">this https URL</a>.

Robotics
Navigating the Docker Ecosystem: A Comprehensive Taxonomy and Survey

Prathamesh Muzumdar,Amol Bhosale,Ganga Prasad Basyal,George Kurian

DOI: https://doi.org/10.9734/AJRCOS/2024/v17i1411

2024-01-03

Abstract:The cloud computing landscape is rapidly expanding and growing in complexity. It has witnessed the emergence of Cloud Computing as a widely adopted model for efficiently processing large volumes of data by harnessing clusters of commodity computers. This evolution enables the handling of massive data through on-demand services, relying on numerous microservices with diverse dependencies. The technology of containers ensures secure storage, allowing for largescale data processing with high scalability and portability. Container technology, particularly exemplified by Docker in the last decade, plays a pivotal role in this scenario. It empowers microservices to process data swiftly, enabling developers to dynamically scale these services in real-time. This paper initiates by establishing a comprehensive taxonomy for delineating container architecture. Focusing specifically on Docker containers, we scrutinize various existing container related literature. Through this taxonomy and survey, we not only discern similarities and disparities in the architectural approaches of Docker container technology but also pinpoint areas necessitating further research.

Distributed, Parallel, and Cluster Computing
Challenges in Docker Development: A Large-scale Study Using Stack Overflow

Mubin Ul Haque,Leonardo Horn Iwaya,M. Ali Babar

DOI: https://doi.org/10.1145/3382494.3410693

2020-08-11

Abstract:Docker technology has been increasingly used among software developers in a multitude of projects. This growing interest is due to the fact that Docker technology supports a convenient process for creating and building containers, promoting close cooperation between developer and operations teams, and enabling continuous software delivery. As a fast-growing technology, it is important to identify the Docker-related topics that are most popular as well as existing challenges and difficulties that developers face. This paper presents a large-scale empirical study identifying practitioners' perspectives on Docker technology by mining posts from the Stack Overflow (SoF) community. Method: A dataset of 113,922 Docker-related posts was created based on a set of relevant tags and contents. The dataset was cleaned and prepared. Topic modelling was conducted using Latent Dirichlet Allocation (LDA), allowing the identification of dominant topics in the domain. Our results show that most developers use SoF to ask about a broad spectrum of Docker topics including framework development, application deployment, continuous integration, web-server configuration and many more. We determined that 30 topics that developers discuss can be grouped into 13 main categories. Most of the posts belong to categories of application development, configuration, and networking. On the other hand, we find that the posts on monitoring status, transferring data, and authenticating users are more popular among developers compared to the other topics. Specifically, developers face challenges in web browser issues, networking error and memory management. Besides, there is a lack of experts in this domain. Our research findings will guide future work on the development of new tools and techniques, helping the community to focus efforts and understand existing trade-offs on Docker topics.

Software Engineering,Information Retrieval
Securing 3rd Party App Integration in Docker-based Cloud Software Ecosystems

Christian Binkowski,Stefan Appel,Andreas Aßmuth

2024-05-18

Abstract:Open software ecosystems are beneficial for customers; they benefit from 3rd party services and applications, e.g. analysis of data using apps, developed and deployed by other companies or open-source communities. One significant advantage of this approach is that other customers may benefit from these newly developed applications as well. Especially software ecosystems utilizing container technologies are prone to certain risks. Docker, in particular, is more vulnerable to attacks than hypervisor based virtualisation as it directly operates on the host system. Docker is a popular representative of containerisation technology which offers a lightweight architecture in order to facilitate the set-up and creation of such software ecosystems. Popular Infrastructure as a Service cloud service providers, like Amazon Web Services or Microsoft Azure, jump on the containerisation bandwagon and provide interfaces for provisioning and managing containers. Companies can benefit from that change of technology and create software ecosystems more efficiently. In this paper, we present a new concept for significant security improvements for cloud-based software ecosystems using Docker for 3rd party app integration. Based on the security features of Docker we describe a secure integration of applications in the cloud environment securely. Our approach considers the whole software lifecycle and includes sandbox testing of potentially dangerous 3rd party apps before these became available to the customers.

Cryptography and Security,Distributed, Parallel, and Cluster Computing
Idiomatic and Reproducible Software Builds using Containers for Reliable Computing

Jonas Weber

DOI: https://doi.org/10.48550/arXiv.1702.02999

2017-02-10

Abstract:Containers as the unit of application delivery are the 'next big thing' in the software development world. They enable developers to create an executable image containing an application bundled with all its dependencies which a user can run inside a controlled environment with virtualized resources. Complex workflows for business-critical applications and research environments require a high degree of reproducibility which can be accomplished using uniquely identified images as units of computation. It will be shown in this thesis that the most widely used approaches to create an image from pre-existing software or from source code lack the ability to provide idiomaticity in their use of the technology as well as proper reproducibility safe-guards. In the first part, existing approaches are formalized and discussed and a new approach is introduced. The approaches are then evaluated using a suite of three different examples. This thesis provides a framework for formalizing operations involving a layered file system, containers and images, and a novel approach to the creation of images using utility containers and layer donning fulfilling the idiomaticity and reproducibility criteria.

Software Engineering
DocLite: A Docker-Based Lightweight Cloud Benchmarking Tool

Blesson Varghese,Lawan Thamsuhang Subba,Long Thai,Adam Barker

DOI: https://doi.org/10.1109/CCGrid.2016.14

2016-03-24

Abstract:Existing benchmarking methods are time consuming processes as they typically benchmark the entire Virtual Machine (VM) in order to generate accurate performance data, making them less suitable for real-time analytics. The research in this paper is aimed to surmount the above challenge by presenting DocLite - Docker Container-based Lightweight benchmarking tool. DocLite explores lightweight cloud benchmarking methods for rapidly executing benchmarks in near real-time. DocLite is built on the Docker container technology, which allows a user-defined memory size and number of CPU cores of the VM to be benchmarked. The tool incorporates two benchmarking methods - the first referred to as the native method employs containers to benchmark a small portion of the VM and generate performance ranks, and the second uses historic benchmark data along with the native method as a hybrid to generate VM ranks. The proposed methods are evaluated on three use-cases and are observed to be up to 91 times faster than benchmarking the entire VM. In both methods, small containers provide the same quality of rankings as a large container. The native method generates ranks with over 90% and 86% accuracy for sequential and parallel execution of an application compared against benchmarking the whole VM. The hybrid method did not improve the quality of the rankings significantly.

Distributed, Parallel, and Cluster Computing
Transform Monolith into Microservices using Docker

Sarita,Sunil Sebastian

DOI: https://doi.org/10.1109/iccubea.2017.8463820

2017-08-01

Abstract:Adaption to container-based micro services architecture will shorten the time to react to the market and help deliver a quality product to the customers. Docker which is based on container-based virtualization has changed the way we look at application deployment. Microservices architecture also introduces complexities which can be mitigated by services scheduling and resource management. In this paper, we present an approach on how to transform your existing monolith into microservices architecture, deploy and deliver using Docker, the way a high velocity organization does with agile development, scalability and an ability to move to the cloud. The contributions of paper are a) An approach to transform existing application into microservices architecture b) Microservices deployment strategy using Docker c) Service discovery pattern d) Proposed strategy to build and ship microservices using Docker
On The Relation Between Outdated Docker Containers, Severity Vulnerabilities and Bugs

Ahmed Zerouali,Tom Mens,Gregorio Robles,Jesus Gonzalez-Barahona

DOI: https://doi.org/10.48550/arXiv.1811.12874

2018-12-01

Abstract:Packaging software into containers is becoming a common practice when deploying services in cloud and other environments. Docker images are one of the most popular container technologies for building and deploying containers. A container image usually includes a collection of software packages, that can have bugs and security vulnerabilities that affect the container health. Our goal is to support container deployers by analysing the relation between outdated containers and vulnerable and buggy packages installed in them. We use the concept of technical lag of a container as the difference between a given container and the most up-to-date container that is possible with the most recent releases of the same collection of packages. For 7,380 official and community Docker images that are based on the Debian Linux distribution, we identify which software packages are installed in them and measure their technical lag in terms of version updates, security vulnerabilities and bugs. We have found, among others, that no release is devoid of vulnerabilities, so deployers cannot avoid vulnerabilities even if they deploy the most recent packages. We offer some lessons learned for container developers in regard to the strategies they can follow to minimize the number of vulnerabilities. We argue that Docker container scan and security management tools should improve their platforms by adding data about other kinds of bugs and include the measurement of technical lag to offer deployers information of when to update.

Software Engineering,Programming Languages
Containerization: Cloud Computing based Inspiration Technology for Adoption through Docker and Kubernetes

S. Rajan,Pradeep Ahirwar,Sanjay Hardikar

DOI: https://doi.org/10.1109/ICESC51422.2021.9532917

2021-08-04

Abstract:The field of cloud computing has been evolving rapidly since inception. Cloud is the virtual pool of resources which can be served to the user through SaaS, PaaS and IaaS flavours. No cloud can be existed without virtualization. With Virtual Machine, the bare metal is virtualized to run multiple Operating System instances. These VMs serve to the users for performing the tasks. All are independent units and user has complete ownership and control to install the required softwares and use as per the wish. The VM solves many problems by optimizing the resources. The developers concern that the code is working fine on the development environment, but fail to work on testing or production environment due to the environment differences, if any. So, the Containerization comes into the picture to address such challenges. In this paper, various aspects of Containerization are explored and highlighted. The Container runtime environment-Docker and Container orchestration tool-Kubernetes are focused and deployed for exploring the possibilities of Containerization adoption, which automate the Container deployment, scaling and load balancing.

Engineering,Computer Science
Container Technologies for ARM Architecture: A Comprehensive Survey of the State-of-the-Art

Shahidullah Kaiser,Turgay Korkmaz,Md. Sadun Haq,Ali Saman Tosun

DOI: https://doi.org/10.1109/access.2022.3197151

IF: 3.9

2022-09-04

IEEE Access

Abstract:Container technology is becoming increasingly popular as an alternative to traditional virtual machines because it provides a faster, lighter, and more portable runtime environment for the applications. A container bundles the application and its binary code, libraries, and configuration files together while sharing the host operating system image. Accordingly, containers efficiently share resources and operate small micro-services, software programs, and even more extensive applications with less overhead than virtual machines. There are many container technologies available with Docker being the most popular and many technologies support multiple architectures, including the ARM architecture. Due to its energy efficiency and high-performance, which are crucial parameters in containerization, ARM architecture is becoming prevalent in container technologies. In this paper, we explore various container technologies that support ARM architecture and investigate the pros and cons of each technology. Moreover, we provide a comparative analysis of both container orchestrators and container runtimes that are most prominent competitors of Docker. We also consider security of container technologies with particular focus on the image scanning tools that supports ARM architecture. Our survey reveals that ARM technology is gaining popularity in containerization and almost all recent technologies support ARM architecture.

computer science, information systems,telecommunications,engineering, electrical & electronic

Docker: lightweight Linux containers for consistent development and deployment

Houdini's Escape

Software Application Development Using Container Technology

An empirical analysis of the Docker container ecosystem on GitHub

Microservice Architecture Practices and Experience: a Focused Look on Docker Configuration Files

Containerization and Orchestration: Implementing OpenShift and Docker

A qualitative and quantitative analysis of container engines

Containers for portable, productive and performant scientific computing

The Rockerverse: Packages and Applications for Containerization with R

Sustainability and Reproducibility via Containerized Computing

Enabling the Deployment of Any-Scale Robotic Applications in Microservice Architectures through Automated Containerization

Navigating the Docker Ecosystem: A Comprehensive Taxonomy and Survey

Challenges in Docker Development: A Large-scale Study Using Stack Overflow

Securing 3rd Party App Integration in Docker-based Cloud Software Ecosystems

Idiomatic and Reproducible Software Builds using Containers for Reliable Computing

DocLite: A Docker-Based Lightweight Cloud Benchmarking Tool

Transform Monolith into Microservices using Docker

On The Relation Between Outdated Docker Containers, Severity Vulnerabilities and Bugs

Containerization: Cloud Computing based Inspiration Technology for Adoption through Docker and Kubernetes

Container Technologies for ARM Architecture: A Comprehensive Survey of the State-of-the-Art