Abstract:Modern software systems like the Linux kernel are among the world's largest and most intricate codebases, continually evolving with new features and increasing complexity. Understanding these systems poses significant challenges due to their scale and the unstructured nature of development artifacts such as commits and mailing list discussions. We introduce Code-Survey, the first LLM-driven methodology designed to systematically explore and analyze large-scale codebases. The central principle behind Code-Survey is to treat LLMs as human participants, acknowledging that software development is also a social activity and thereby enabling the application of established social science techniques. By carefully designing surveys, Code-Survey transforms unstructured data, such as commits, emails, into organized, structured, and analyzable datasets. This enables quantitative analysis of complex software evolution and uncovers valuable insights related to design, implementation, maintenance, reliability, and security. To demonstrate the effectiveness of Code-Survey, we apply it to the Linux kernel's eBPF subsystem. We construct the Linux-bpf dataset, comprising over 670 features and 16,000 commits from the Linux community. Our quantitative analysis uncovers important insights into the evolution of eBPF, such as development patterns, feature interdependencies, and areas requiring attention for reliability and security. The insights have been initially validated by eBPF experts. Furthermore, Code-Survey can be directly applied to other subsystems within Linux and to other large-scale software projects. By providing a versatile tool for systematic analysis, Code-Survey facilitates a deeper understanding of complex software systems, enabling improvements across a variety of domains and supporting a wide range of empirical studies. The code and dataset is open-sourced.

World of Code: An Infrastructure for Mining the Universe of Open Source VCS Data

World of Code: Enabling a Research Workflow for Mining and Analyzing the Universe of Open Source VCS data

A Large-Scale Empirical Study of Open Source License Usage: Practices and Challenges

Building the Collaboration Graph of Open-Source Software Ecosystem

Constructing Temporal Networks of OSS Programming Language Ecosystems

OSS License Identification at Scale: A Comprehensive Dataset Using World of Code

Evolving Collaboration, Dependencies, and Use in the Rust Open Source Software Ecosystem

Towards a Critical Open-Source Software Database

The Code the World Depends On: A First Look at Technology Makers' Open Source Software Dependencies

Open Source Software Sustainability: Combining Institutional Analysis and Socio-Technical Networks

Quantitative analysis of open source project in GitHub community based on big data

OpenDigger: Data Mining and Information Service System for Open Collaboration Digital Ecosystem

Sourcerer: An infrastructure for large-scale collection and analysis of open-source code

Review code evolution history in OSS universe

Code-Survey: An LLM-Driven Methodology for Analyzing Large-Scale Codebases

Dataset: Copy-based Reuse in Open Source Software

An Analysis of Open Source Operating Systems Based on Complex Networks Theory

Code Architecture and Open Source Software Development.

The Geography of Open Source Software: Evidence from GitHub

Mining the Contributions Along the Lifecycles of Open-Source Projects

Measuring Software Innovation with Open Source Software Development Data