Abstract:Background: Continuous Integration (CI) systems are now the bedrock of several software development practices. Several tools such as TravisCI, CircleCI, and Hudson, that implement CI practices, are commonly adopted by software engineers. However, the way that software engineers use these tools could lead to what we call "Continuous Integration Theater", a situation in which software engineers do not employ these tools effectively, leading to unhealthy CI practices. Aims: The goal of this paper is to make sense of how commonplace are these unhealthy continuous integration practices being employed in practice. Method: By inspecting 1,270 open-source projects that use TravisCI, the most used CI service, we quantitatively studied how common is to use CI (1) with infrequent commits, (2) in a software project with poor test coverage, (3) with builds that stay broken for long periods, and (4) with builds that take too long to run. Results: We observed that 748 ($sim$60%) projects face infrequent commits, which essentially makes the merging process harder. Moreover, we were able to find code coverage information for 51 projects. The average code coverage was 78%, although Ruby projects have a higher code coverage than Java projects (86% and 63%, respectively). However, some projects with very small coverage ($sim$4%) were found. Still, we observed that 85% of the studied projects have at least one broken build that take more than four days to be fixed. Interestingly, very small projects (up to 1,000 lines of code) are the ones that take the longest to fix broken builds. Finally, we noted that, for the majority of the studied projects, the build is executed under the 10 minutes rule of thumb. Conclusions: Our results are important to an increasing community of software engineers that employ CI practices on daily basis but may not be aware of bad practices that are eventually employed.

The impact of a continuous integration service on the delivery time of merged pull requests

The Impact of Continuous Integration on Other Software Development Practices: A Large-Scale Empirical Study

Investigating the Impact of Continuous Integration Practices on the Productivity and Quality of Open-Source Projects

Uncovering the Benefits and Challenges of Continuous Integration Practices

Continuous Integration Theater

The effects of continuous integration on software development: a systematic literature review

Initial and Eventual Software Quality Relating to Continuous Integration in GitHub

Analyzing the Effects of CI/CD on Open Source Repositories in GitHub and GitLab

How do Machine Learning Projects use Continuous Integration Practices? An Empirical Study on GitHub Actions

Quality and productivity outcomes relating to continuous integration in GitHub

On the Need to Monitor Continuous Integration Practices -- An Empirical Study

Chronicles of CI/CD: A Deep Dive into its Usage Over Time

Do Small Code Changes Merge Faster? A Multi-Language Empirical Investigation

DevOps Improvements for Reduced Cycle Times with Integrated Test Optimizations for Continuous Integration

On the usage, co-usage and migration of CI/CD tools: A qualitative analysis

The Why, When, What, and How about Predictive Continuous Integration: A Simulation-Based Investigation

Continuous Integration and Software Quality: A Causal Explanatory Study

Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges and Practices

TravisTorrent: Synthesizing Travis CI and GitHub for Full-Stack Research on Continuous Integration

Mining Code Review Data to Understand Waiting Times Between Acceptance and Merging: An Empirical Analysis

Effectiveness Of Code Contribution : From Patch-Based To Pull-Request-Based Tools