ChatDBG: An AI-Powered Debugging Assistant

Kyla Levin,Nicolas van Kempen,Emery D. Berger,Stephen N. Freund

2024-09-24

Abstract:Debugging is a critical but challenging task for programmers. This paper proposes ChatDBG, an AI-powered debugging assistant. ChatDBG integrates large language models (LLMs) to significantly enhance the capabilities and user-friendliness of conventional debuggers. ChatDBG lets programmers engage in a collaborative dialogue with the debugger, allowing them to pose complex questions about program state, perform root cause analysis for crashes or assertion failures, and explore open-ended queries like `why is x null?'. To handle these queries, ChatDBG grants the LLM autonomy to "take the wheel": it can act as an independent agent capable of querying and controlling the debugger to navigate through stacks and inspect program state. It then reports its findings and yields back control to the programmer. Our ChatDBG prototype integrates with standard debuggers including LLDB and GDB for native code and Pdb for Python. Our evaluation across a diverse set of code, including C/C++ code with known bugs and a suite of Python code including standalone scripts and Jupyter notebooks, demonstrates that ChatDBG can successfully analyze root causes, explain bugs, and generate accurate fixes for a wide range of real-world errors. For the Python programs, a single query led to an actionable bug fix 67% of the time; one additional follow-up query increased the success rate to 85%. ChatDBG has seen rapid uptake; it has already been downloaded roughly 50,000 times.

Software Engineering,Artificial Intelligence,Machine Learning,Programming Languages

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper proposes an AI-driven debugging assistant named **ChatDBG**, aimed at addressing the challenges programmers face during the debugging process. Specifically, ChatDBG integrates large language models (LLMs), significantly enhancing the functionality and usability of traditional debuggers. It allows programmers to engage in collaborative dialogues with the debugger, posing complex questions about the program state, performing root cause analysis to handle crashes or assertion failures, and exploring open-ended queries (e.g., "Why is x null?"). To handle these queries, ChatDBG grants autonomy to the LLM, enabling it to operate independently, query and control the debugger to navigate the stack and inspect the program state, and report the findings back to the programmer.

### Main Contributions

1. **Introduction of ChatDBG**: The first AI-driven debugging assistant.
2. **Description of ChatDBG Prototype Implementation**: Integrated with standard debuggers such as GDB, LLDB, and Pdb.
3. **Proposal of the "Autonomous Control" Approach**: Combining agent reasoning in large language models with developer tools.
4. **Evaluation of ChatDBG**: Demonstrating its advantages over existing debugging functionalities.

### Example Debugging Session

The paper illustrates the workflow of ChatDBG through a concrete example. In a Python script containing multiple errors, ChatDBG successfully diagnosed and fixed the defects, improving debugging efficiency. Users can ask questions in natural language, and ChatDBG utilizes the LLM to conduct in-depth analysis and provide suggestions.

In summary, the paper aims to enhance the efficiency and accuracy of the debugging process through ChatDBG, particularly for novice programmers who often lack experience in effectively using debuggers. At the same time, for experienced programmers, ChatDBG can enhance their debugging capabilities through natural dialogue.