Bug Analysis in Jupyter Notebook Projects: An Empirical Study

Taijara Loiola de Santana,Paulo Anselmo da Mota Silveira Neto,Eduardo Santana de Almeida,Iftekhar Ahmed

DOI: https://doi.org/10.1145/3641539

IF: 3.685

2024-01-22

ACM Transactions on Software Engineering and Methodology

Abstract:Computational notebooks, such as Jupyter, have been widely adopted by data scientists to write code for analyzing and visualizing data. Despite their growing adoption and popularity, few studies were found to understand Jupyter development challenges from the practitioners’ point of view. This paper presents a systematic study of bugs and challenges that Jupyter practitioners face through a large-scale empirical investigation. We mined 14,740 commits from 105 GitHub open-source projects with Jupyter notebook code. Next, we analyzed 30,416 Stack Overflow posts, which gave us insights into bugs that practitioners face when developing Jupyter notebook projects. Next, we conducted nineteen interviews with data scientists to uncover more details about Jupyter bugs and to gain insight into Jupyter developers’ challenges. Finally, to validate the study results and proposed taxonomy, we conducted a survey with 91 data scientists. We also highlight bug categories, their root causes, and the challenges that Jupyter practitioners face.

computer science, software engineering

What problem does this paper attempt to address?

The paper aims to address issues related to errors and challenges encountered in Jupyter Notebook projects. Specifically, researchers conducted a large-scale empirical survey, which included the analysis of 14,740 commit records from 105 GitHub open-source projects, and 30,416 posts on Stack Overflow, to identify and categorize the common types of errors, their root causes, and the practical impact of these errors on data scientists. In addition, the study included 19 semi-structured interviews to gain a deeper understanding of how errors affect the daily work of data scientists and to further validate the research findings. Finally, to verify the research results and the proposed error classification system, the researchers surveyed 91 data scientists.

The core questions the research focused on are:

1. What types of errors occur most frequently in Jupyter Notebook projects?
2. What are the root causes of these errors?
3. What are the common impacts of errors on projects?
4. What challenges do data scientists face when using Jupyter projects in practice?

The researchers proposed a comprehensive error classification system, which includes eight different categories of errors, specifically for Jupyter Notebook projects. This system aims to provide researchers and practitioners with an in-depth understanding of error types, root causes, and impacts, thereby helping to develop new tools for detecting errors, managing error reports, locating potential error sites, suggesting possible fixes, and monitoring and improving code quality during the development process.