Abstract:Computational notebooks have become the primary coding environment for data scientists. However, research on their code quality is still emerging, and the code shared is often of poor quality. Given the importance of maintenance and reusability, understanding the metrics that affect notebook code comprehensibility is crucial. Code understandability, a qualitative variable, is closely tied to user opinions. Traditional approaches to measuring it either use limited questionnaires to review a few code pieces or rely on metadata such as likes and votes in software repositories. Our approach enhances the measurement of Jupyter notebook understandability by leveraging user comments related to code understandability. As a case study, we used 542,051 Kaggle Jupyter notebooks from our previous research, named DistilKaggle. We employed a fine-tuned DistilBERT transformer to identify user comments associated with code understandability. We established a criterion called User Opinion Code Understandability (UOCU), which considers the number of relevant comments, upvotes on those comments, total notebook views, and total notebook upvotes. UOCU proved to be more effective than previous methods. Furthermore, we trained machine learning models to predict notebook code understandability based solely on their metrics. We collected 34 metrics for 132,723 final notebooks as features in our dataset, using UOCU as the label. Our predictive model, using the Random Forest classifier, achieved 89% accuracy in predicting the understandability levels of computational notebooks.

Make It Make Sense! Understanding and Facilitating Sensemaking in Computational Notebooks

What's Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities

Notable: On-the-fly Assistant for Data Storytelling in Computational Notebooks

Notable: On-the-fly assistant for data storytelling in computational notebooks

WHATSNEXT: Guidance-enriched Exploratory Data Analysis with Interactive, Low-Code Notebooks

Facilitating Mixed-Methods Analysis with Computational Notebooks

StickyLand: Breaking the Linear Presentation of Computational Notebooks

Loops: Leveraging Provenance and Visualization to Support Exploratory Data Analysis in Notebooks

Notably Inaccessible -- Data Driven Understanding of Data Science Notebook (In)Accessibility

Predicting the Understandability of Computational Notebooks through Code Metrics Analysis

Talking datasets: Understanding data sensemaking behaviours

Understanding Students' Problem-Solving Strategies in a Synergistic Learning-by-Modeling Environment.

A System for Image Understanding using Sensemaking and Narrative

SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks

Towards Transparent, Reusable, and Customizable Data Science in Computational Notebooks

NBSearch: Semantic Search and Visual Exploration of Computational Notebooks

Assessing the Quality of Computational Notebooks for a Frictionless Transition from Exploration to Production

Teaching machines to understand data science code by semantic enrichment of dataflow graphs

Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models

Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks

Designing for Collaborative Sensemaking: Leveraging Human Cognition For Complex Tasks