Abstract:Context: In continuous deployment, software and services are rapidly deployed to end-users using an automated deployment pipeline. Defects in infrastructure as code (IaC) scripts can hinder the reliability of the automated deployment pipeline. We hypothesize that certain properties of IaC source code such as lines of code and hard-coded strings used as configuration values, show correlation with defective IaC scripts.Objective: The objective of this paper is to help practitioners in increasing the quality of infrastructure as code (IaC) scripts through an empirical study that identifies source code properties of defective IaC scripts.Methodology: We apply qualitative analysis on defect-related commits mined from open source software repositories to identify source code properties that correlate with defective IaC scripts. Next, we survey practitioners to assess the practitioner's agreement level with the identified properties. We also construct defect prediction models using the identified properties for 2,439 scripts collected from four datasets.Results: We identify 10 source code properties that correlate with defective IaC scripts. Of the identified 10 properties we observe lines of code and hard-coded string i.e. putting strings as configuration values, to show the strongest correlation with defective IaC scripts. According to our survey analysis, majority of the practitioners show agreement for two properties: include, the property of executing external modules or scripts, and hard-coded string. Using the identified properties, our constructed defect prediction models show a precision of 0.70<math>∼</math>0.78, and a recall of 0.54<math>∼</math>0.67.Conclusion: Based on our findings, we recommend practitioners to allocate sufficient inspection and testing efforts on IaC scripts that include any of the identified 10 source code properties of IaC scripts.

Quality Assurance for Infrastructure Orchestrators: Emerging Results from Ansible

Practitioner Perceptions of Ansible Test Smells.

Where Are The Gaps? A Systematic Mapping Study of Infrastructure as Code Research

Source code properties of defective infrastructure as code scripts

Security Smells in Ansible and Chef Scripts: A Replication Study

Exploring LLM-based Automated Repairing of Ansible Script in Edge-Cloud Infrastructures

Research on Automated Operation and Maintenance Methods for Power Management Networks Based on Ansible

FindICI: Using machine learning to detect linguistic inconsistencies between code and natural language descriptions in infrastructure-as-code

LLM and Infrastructure as a Code use case

Exploring Security Practices in Infrastructure as Code: An Empirical Study

GLITCH: Automated Polyglot Security Smell Detection in Infrastructure as Code

Can We Use Authentic Learning to Educate Students about Secure Infrastructure As Code Development?

Continuous Intrusion: Characterizing the Security of Continuous Integration Services.

A Qualitative Study of Architectural Design Issues in DevOps

Initial and Eventual Software Quality Relating to Continuous Integration in GitHub

State-Of-The-Practice in Quality Assurance in Java-Based Open Source Software Development

A Roadmap for Software Testing in Open Collaborative Development Environments

An action research study of quality improvement in instrument packaging procedures for the central sterile supply department

Automating Quality Control on a Shoestring, a Case Study

Orchestration or Automation: Authentication Flaw Detection in Android Apps

An Ethnographic Study on the CI of A Large Scale Project