Towards Practical and Useful Automated Program Repair for Debugging

Qi Xin,Haojun Wu,Steven P. Reiss,Jifeng Xuan

2024-07-12

Abstract:Current automated program repair (APR) techniques are far from being practical and useful enough to be considered for realistic debugging. They rely on unrealistic assumptions including the requirement of a comprehensive suite of test cases as the correctness criterion and frequent program re-execution for patch validation; they are not fast; and their ability of repairing the commonly arising complex bugs by fixing multiple locations of the program is very limited. We hope to substantially improve APR's practicality, effectiveness, and usefulness to help people debug. Towards this goal, we envision PracAPR, an interactive repair system that works in an Integrated Development Environment (IDE) to provide effective repair suggestions for debugging. PracAPR does not require a test suite or program re-execution. It assumes that the developer uses an IDE debugger and the program has suspended at a location where a problem is observed. It interacts with the developer to obtain a problem specification. Based on the specification, it performs test-free, flow-analysis-based fault localization, patch generation that combines large language model-based local repair and tailored strategy-driven global repair, and program re-execution-free patch validation based on simulated trace comparison to suggest repairs. By having PracAPR, we hope to take a significant step towards making APR useful and an everyday part of debugging.

Software Engineering

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the lack of practicality and effectiveness of current Automated Program Repair (APR) techniques in actual debugging. Specifically, the paper points out that current APR techniques face the following three main challenges:

1. **Unrealistic assumptions**:
- Current APR techniques rely on unrealistic assumptions. For example, a comprehensive test case set is required as a criterion for correctness, and the program needs to be frequently re - executed to verify patches. However, in actual debugging scenarios, it is usually not possible to assume the existence of a high - quality test case set, especially in the early stages of software development. In addition, frequent program re - execution makes APR not fast enough.

2. **Weak local repair ability**:
- Although APR has made certain progress in generating patches for a single program location, its repair ability is still weak. According to the evaluation of existing tools, traditional non - learning methods can only repair a small number (less than 23%) of single - block errors. Even learning methods using large - language models (LLM), although improved, the state - of - the - art methods such as Repilot still fail to repair 57.3% of single - location errors. In addition, existing techniques may also generate over - fitted patches, which may have a negative impact on debugging.

3. **Limited global repair ability**:
- Current APR techniques perform poorly in dealing with complex multi - location errors. Multi - location errors refer to repairs that require changes to multiple locations in the program. Research shows that at least 40% of real - error repairs involve multi - location errors. However, existing techniques are less effective in repairing such errors and can at most repair 8 single - fault multi - location errors.

To address these challenges, the paper proposes a new interactive repair system, PracAPR, aiming to improve the practicality and effectiveness of APR and make it a part of daily debugging. The main features of PracAPR include:

- **No need for test cases and program re - execution**: PracAPR runs in an Integrated Development Environment (IDE) and works in cooperation with the IDE debugger. It assumes that the program pauses at the location where the problem is found. It obtains the problem description through interaction with the developer, and then performs test - free fault location, patch generation, and patch verification based on simulated trajectory comparison.

- **Combination of local and global repair strategies**: PracAPR uses large - language models (LLM) for local repair and combines multiple customized strategies for global repair to deal with complex multi - location errors.

Through these improvements, PracAPR hopes to significantly enhance the practicality and effectiveness of APR and make it a powerful tool in the debugging process.