Linking Code and Documentation Churn: Preliminary Analysis

Ani Hovhannisyan,Youmei Fan,Gema Rodriguez-Perez,Raula Gaikovina Kula

2024-10-08

Abstract:Code churn refers to the measure of the amount of code added, modified, or deleted in a project and is often used to assess codebase stability and maintainability. Program comprehension or how understandable the changes are, is equally important for maintainability. Documentation is crucial for knowledge transfer, especially when new maintainers take over abandoned code. We emphasize the need for corresponding documentation updates, as this reflects project health and trustworthiness as a third-party library. Therefore, we argue that every code change should prompt a documentation update (defined as documentation churn). Linking code churn changes with documentation updates is important for project sustainability, as it facilitates knowledge transfer and reduces the effort required for program comprehension. This study investigates the synchrony between code churn and documentation updates in three GitHub open-source projects. We will use qualitative analysis and repository mining to examine the alignment and correlation of code churn and documentation updates over time. We want to identify which code changes are likely synchronized with documentation and to what extent documentation can be auto-generated. Preliminary results indicate varying degrees of synchrony across projects, highlighting the importance of integrated concurrent documentation practices and providing insights into how recent technologies like AI, in the form of Large Language Models (i.e., LLMs), could be leveraged to keep code and documentation churn in sync. The novelty of this study lies in demonstrating how synchronizing code changes with documentation updates can improve the development lifecycle by enhancing diversity and efficiency.

Software Engineering

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the synchronization between code churn and document updates. Specifically, the author focuses on whether the addition, modification, or deletion of code (i.e., code churn) can be reflected in the corresponding documents in a timely and accurate manner during the software development process. This problem is crucial for the maintainability and sustainable development of the project, because good documentation not only helps new maintainers quickly understand the project, but also improves the readability of the code and the credibility of the project.

### Main research questions:
1. **Synchronization between code churn and document updates**: The research aims to explore the degree of synchronization between code churn and document updates, especially in three GitHub open - source projects.
2. **Possibility of automated document generation**: Explore how to use modern technologies such as large - language models (LLMs) to automatically generate documents, in order to reduce the manual workload of developers and improve the quality and timeliness of documents.
3. **Impact of diversity and inclusion**: The research also focuses on the impact of document quality on the diversity and inclusion of the project, especially how to attract more contributors from different backgrounds by improving the documents.

### Research background:
- **Code Churn**: Refers to the amount of code added, modified, or deleted in a project, and is usually used to evaluate the stability and maintainability of the code base.
- **Document Churn**: Refers to the frequency and quality of document updates related to code churn, which is very important for knowledge transfer and project health.
- **Existing challenges**: Although existing research and technology have attempted to solve the problem of code - document synchronization, in actual projects, there is still a significant gap between code churn and document updates, resulting in increased maintenance costs and decreased software quality.

### Research methods:
- **Quantitative analysis**: By mining GitHub repository data, collect statistical information on code churn and document updates.
- **Qualitative analysis**: Manually classify selected code churns, analyze their nature and importance, especially the association between code churn and document updates.

### Preliminary results:
Preliminary results show that there are significant differences in the synchronization between code churn and document updates in the three projects studied. For example, in the HTTP project, there are 14,768 code churns, but only 5,417 are synchronized with document updates, indicating a large knowledge gap.

### Future work:
- **Expand the data set**: Expand the research scope to more projects, such as PyPI, NPM, and GNU libraries, and analyze developer activities and their relationship with document updates.
- **AI - assisted strategies**: Explore the use of AI technologies such as large - language models (LLMs) to automatically generate documents to improve the efficiency and quality of document updates.
- **In - depth analysis of influencing factors**: Research why some contributors ignore document updates and explore possible improvement strategies.

In conclusion, the core problem of this paper is to propose methods for improving document practices by studying the synchronization between code churn and document updates, thereby enhancing the maintainability and sustainability of the project.