Context-Aware Code Change Embedding for Better Patch Correctness Assessment

Bo Lin,Shangwen Wang,Ming Wen,Xiaoguang Mao

DOI: https://doi.org/10.1145/3505247

IF: 3.685

2022-05-18

ACM Transactions on Software Engineering and Methodology

Abstract:Despite the capability in successfully fixing more and more real-world bugs, existing

computer science, software engineering

What problem does this paper attempt to address?

TASK CONTEXT: A Tool for Predicting Code Context Models for Software Development Tasks

Yifeng Wang,Yuhang Lin,Zhiyuan Wan,Xiaohu Yang

DOI: https://doi.org/10.1109/ICSE-Companion58688.2023.00045

2023-01-01

Abstract:A code context model consists of code elements and their relations relevant to a development task. Previous studies found that the explicit formation of code context models can benefit software development practices, e.g., code navigation and searching. However, little focus has been put on how to proactively form code context models. In this paper, we propose a tool named TASK CONTEXT for predicting code context models and implement it as an Eclipse plug-in. TASK CONTEXT uses the abstract topological patterns of how developers investigate structurally connected code elements when performing tasks. The tool captures the code elements navigated and searched by a developer to construct an initial code context model. The tool then applies abstract topological patterns with the initial code context model as input and recommends code elements up to 3 steps away in the code structure from the initial code context model. The experimental results indicate that our approach can predict code context models effectively, with a significantly higher F-measure than the state-of-the-art (0.57 over 0.23 on average). Furthermore, the user study suggests that our tool can help practitioners complete development tasks faster and more often as compared to standard Eclipse mechanism.
Context-Aware Patch Generation for Better Automated Program Repair

Ming Wen,Junjie Chen,Rongxin Wu,Dan Hao,Shing-Chi Cheung

DOI: https://doi.org/10.1145/3180155.3180233

2018-01-01

Abstract:The effectiveness of search-based automated program repair is limited in the number of correct patches that can be successfully generated. There are two causes of such limitation. First, the search space does not contain the correct patch. Second, the search space is huge and therefore the correct patch cannot be generated (ie correct patches are either generated after incorrect plausible ones or not generated within the time budget). To increase the likelihood of including the correct patches in the search space, we propose to work at a fine granularity in terms of AST nodes. This, however, will further enlarge the search space, increasing the challenge to find the correct patches. We address the challenge by devising a strategy to prioritize the candidate patches based on their likelihood of being correct. Specifically, we study the use of AST nodes' context information to estimate the likelihood. In this paper, we propose CapGen, a context-aware patch generation technique. The novelty which allows CapGen to produce more correct patches lies in three aspects: (1) The fine-granularity design enables it to find more correct fixing ingredients; (2) The context-aware prioritization of mutation operators enables it to constrain the search space; (3) Three context-aware models enable it to rank correct patches at high positions before incorrect plausible ones. We evaluate CapGen on Defects4J and compare it with the state-of-the-art program repair techniques. Our evaluation shows that CapGen outperforms and complements existing techniques. CapGen achieves a high precision of 84.00% and can prioritize the correct patches before 98.78% of the incorrect plausible ones.
Fine-grained Commit-level Vulnerability Type Prediction by CWE Tree Structure.

Shengyi Pan,Lingfeng Bao,Xin Xia,David Lo,Shanping Li

DOI: https://doi.org/10.1109/icse48619.2023.00088

2023-01-01

Abstract:Identifying security patches via code commits to allow early warnings and timely fixes for Open Source Software (OSS) has received increasing attention. However, the existing detection methods can only identify the presence of a patch (i.e., a binary classification) but fail to pinpoint the vulnerability type. In this work, we take the first step to categorize the security patches into fine-grained vulnerability types. Specifically, we use the Common Weakness Enumeration (CWE) as the label and perform fine-grained classification using categories at the third level of the CWE tree. We first formulate the task as a Hierarchical Multi-label Classification (HMC) problem, i.e., inferring a path (a sequence of CWE nodes) from the root of the CWE tree to the node at the target depth. We then propose an approach named TREEVUL with a hierarchical and chained architecture, which manages to utilize the structure information of the CWE tree as prior knowledge of the classification task. We further propose a tree structure aware and beam search based inference algorithm for retrieving the optimal path with the highest merged probability. We collect a large security patch dataset from NVD, consisting of 6,541 commits from 1,560 GitHub OSS repositories. Experimental results show that TREEVUL significantly outperforms the best performing baselines, with improvements of 5.9%, 25.0%, and 7.7% in terms of weighted F1-score, macro F1-score, and MCC, respectively. We further conduct a user study and a case study to verify the practical value of TREEVUL in enriching the binary patch detection results and improving the data quality of NVD, respectively.
Patchmatch: A Tool for Locating Patches of Open Source Project Vulnerabilities

Kedi Shen,Yun Zhang,Lingfeng Bao,Zhiyuan Wan,Zhuorong Li,Minghui Wu

DOI: https://doi.org/10.1109/ICSE-COMPANION58688.2023.00049

2023-01-01

Abstract:With the rapid development of open source projects, the continuous emergence of vulnerabilities in the project brings great challenges to the security of the project. Security patches are one of the best ways to deal with vulnerabilities, but are not well applied currently. Although there are sites like CVE/NVD that provide information about vulnerabilities, many of the vulnerabilities disclosed by CVE/NVD are not accompanied by security patches. This makes it difficult for developers to apply patches. In the present study, a sorting method based on extracting multidimensional features from auxiliary information in CVE/NVD was proposed. And we made a further step, we proposed VCMATCH, a model for mining semantic information in vulnerability description and code commit messages, which has good recall rate and applicability across projects. On this basis, we established Patchmatch, a tool for helping developers to quickly locate patches. Given a vulnerability, Patchmatch can forecast the implicit patches in the code repository's commits. Patchmatch also has a visual webpage for information statistics and a display web page to help developers manage all kinds of information in the code repository. A demo video of Patchmatch is at https://www.youtube.com/watch?v=nOBSMFtZV8A. Patchmatch is in https://github.com/Sklud1456/patchmatch.
Just-In-Time Defect Identification and Localization: A Two-Phase Framework.

Meng Yan,Xin Xia,Yuanrui Fan,Ahmed E. Hassan,David Lo,Shanping Li

DOI: https://doi.org/10.1109/tse.2020.2978819

IF: 7.4

2022-01-01

IEEE Transactions on Software Engineering

Abstract:Defect localization aims to locate buggy program elements (e.g., buggy files, methods or lines of code) based on defect symptoms, e.g., bug reports or program spectrum. However, when we receive the defect symptoms, the defect has been exposed and negative impacts have been introduced. Thus, one challenging task is: whether we can locate buggy program prior to the appearance of the defect symptom (e.g., when buggy program elements are being committed to a version control system). We refer to this type of defect localization as “Just-In-Time (JIT) Defect localization”. Although many prior studies have proposed various JIT defect identification methods to identify whether a new change is buggy, these prior methods do not locate the suspicious positions. Thus, JIT defect localization is the next step of JIT defect identification (i.e., after a buggy change is identified, suspicious source code lines are located). To address this problem, we propose a two-phase framework, i.e., JIT defect identification and JIT defect localization. Given a new change, JIT defect identification will identify it as buggy change or clean change first. If a new change is identified as buggy, JIT defect localization will rank the source code lines introduced by the new change according to their suspiciousness scores. The source code lines ranked at the top of the list are estimated as the defect location. For JIT defect identification phase, we use 14 change-level features to build a classifier by following existing approach. For JIT defect localization phase, we propose a JIT defect localization approach that leverages software naturalness with the N-gram model. To evaluate the proposed framework, we conduct an empirical study on 14 open source projects with a total of 177,250 changes. The results show that software naturalness is effective for our JIT defect localization. Our model achieves a reasonable performance, and outperforms the two baselines (i.e., random guess and a static bug finder (i.e., PMD)) by a substantial margin in terms of four ranking measures.
The Inversive Relationship Between Bugs and Patches: An Empirical Study

Jinhan Kim,Jongchan Park,Shin Yoo

2023-03-01

Abstract:Software bugs pose an ever-present concern for developers, and patching such bugs requires a considerable amount of costs through complex operations. In contrast, introducing bugs can be an effortless job, in that even a simple mutation can easily break the Program Under Test (PUT). Existing research has considered these two opposed activities largely separately, either trying to automatically generate realistic patches to help developers, or to find realistic bugs to simulate and prevent future defects. Despite the fundamental differences between them, however, we hypothesise that they do not syntactically differ from each other when considered simply as code changes. To examine this assumption systematically, we investigate the relationship between patches and buggy commits, both generated manually and automatically, using a clustering and pattern analysis. A large scale empirical evaluation reveals that up to 70% of patches and faults can be clustered together based on the similarity between their lexical patterns; further, 44% of the code changes can be abstracted into the identical change patterns. Moreover, we investigate whether code mutation tools can be used as Automated Program Repair (APR) tools, and APR tools as code mutation tools. In both cases, the inverted use of mutation and APR tools can perform surprisingly well, or even better, when compared to their original, intended uses. For example, 89% of patches found by SequenceR, a deep learning based APR tool, can also be found by its inversion, i.e., a model trained with faults and not patches. Similarly, real fault coupling study of mutants reveals that TBar, a template based APR tool, can generate 14% and 3% more fault couplings than traditional mutation tools, PIT and Major respectively, when used as a mutation tool.

Software Engineering
Neural Program Repair with Program Dependence Analysis and Effective Filter Mechanism

Yuwei Zhang,Ge Li,Zhi Jin,Ying Xing

DOI: https://doi.org/10.48550/arxiv.2305.09315

2023-01-01

Abstract:Automated program repair is a crucial task for improving the efficiency of software developers. Recently, neural-based techniques have demonstrated significant promise in generating correct patches for buggy code snippets. However, most existing approaches arbitrarily treat the buggy context without any analysis to capture the semantic relationship between the buggy statement and its context. Additionally, we observe that existing neural models may output an unaltered patch consistent with the input buggy code snippet, which fails to be the correct human-written one for fixing the given bug. To address the aforementioned limitations, we present in this paper a novel neural program repair framework called \approach, which adapts the general pre-trained language model for fixing single-line Java bugs. We make the first attempt to use program slicing to extract contextual information directly related to the given buggy statement as repair ingredients from the corresponding program dependence graph and eliminate unaltered patches using an intuitive but effective filter mechanism. We demonstrate the effectiveness of \approach on five benchmarks when compared with state-of-the-art baselines.
Patch Space Exploration using Static Analysis Feedback

Yuntong Zhang,Andreea Costea,Ridwan Shariffdeen,Davin McCall,Abhik Roychoudhury

2023-08-01

Abstract:Automated Program Repair (APR) techniques typically rely on a given test-suite to guide the repair process. Apart from the need to provide test oracles, this makes the produced patches prone to test data over-fitting. In this work, instead of relying on test cases, we show how to automatically repair memory safety issues, by leveraging static analysis (specifically Incorrectness Separation Logic) to guide repair. Our proposed approach learns what a desirable patch is by inspecting how close a patch is to fixing the bug based on the feedback from incorrectness separation logic based static analysis (specifically the Pulse analyser), and turning this information into a distribution of probabilities over context free grammars. Furthermore, instead of focusing on heuristics for reducing the search space of patches, we make repair scalable by creating classes of equivalent patches according to the effect they have on the symbolic heap, and then invoking the validation oracle only once per class of patch equivalence. This allows us to efficiently discover repairs even in the presence of a large pool of patch candidates offered by our generic patch synthesis mechanism. Experimental evaluation of our approach was conducted by repairing real world memory errors in OpenSSL, swoole and other subjects. The evaluation results show the scalability and efficacy of our approach in automatically producing high quality patches.

Software Engineering
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation

Michele Tufano,Cody Watson,Gabriele Bavota,Massimiliano Di Penta,Martin White,Denys Poshyvanyk

DOI: https://doi.org/10.48550/arXiv.1812.08693

2019-05-21

Abstract:Millions of open-source projects with numerous bug fixes are available in code repositories. This proliferation of software development histories can be leveraged to learn how to fix common programming bugs. To explore such a potential, we perform an empirical study to assess the feasibility of using Neural Machine Translation techniques for learning bug-fixing patches for real defects. First, we mine millions of bug-fixes from the change histories of projects hosted on GitHub, in order to extract meaningful examples of such bug-fixes. Next, we abstract the buggy and corresponding fixed code, and use them to train an Encoder-Decoder model able to translate buggy code into its fixed version. In our empirical investigation we found that such a model is able to fix thousands of unique buggy methods in the wild. Overall, this model is capable of predicting fixed patches generated by developers in 9-50% of the cases, depending on the number of candidate patches we allow it to generate. Also, the model is able to emulate a variety of different Abstract Syntax Tree operations and generate candidate patches in a split second.

Software Engineering
Shaping Program Repair Space with Existing Patches and Similar Code

Jiajun Jiang,Yingfei Xiong,Hongyu Zhang,Qing Gao,Xiangqun Chen

DOI: https://doi.org/10.1145/3213846.3213871

2018-01-01

Abstract:Automated program repair (APR) has great potential to reduce bug-fixing effort and many approaches have been proposed in recent years. APRs are often treated as a search problem where the search space consists of all the possible patches and the goal is to identify the correct patch in the space. Many techniques take a data-driven approach and analyze data sources such as existing patches and similar source code to help identify the correct patch. However, while existing patches and similar code provide complementary information, existing techniques analyze only a single source and cannot be easily extended to analyze both. In this paper, we propose a novel automatic program repair approach that utilizes both existing patches and similar code. Our approach mines an abstract search space from existing patches and obtains a concrete search space by differencing with similar code snippets. Then we search within the intersection of the two search spaces. We have implemented our approach as a tool called SimFix, and evaluated it on the Defects4J benchmark. Our tool successfully fixed 34 bugs. To our best knowledge, this is the largest number of bugs fixed by a single technology on the Defects4J benchmark. Furthermore, as far as we know, 13 bugs fixed by our approach have never been fixed by the current approaches.
PatchZero: Zero-Shot Automatic Patch Correctness Assessment

Xin Zhou,Bowen Xu,Kisub Kim,DongGyun Han,Thanh Le-Cong,Junda He,Bach Le,David Lo

2024-03-22

Abstract:Automated Program Repair (APR) techniques have shown more and more promising results in fixing real-world bugs. Despite the effectiveness, APR techniques still face an overfitting problem: a generated patch can be incorrect although it passes all tests. It is time-consuming to manually evaluate the correctness of generated patches that can pass all tests. To address this problem, many approaches have been proposed to automatically assess the correctness of patches generated by APR techniques. These approaches are mainly evaluated within the cross-validation setting. However, for patches generated by a new or unseen APR tool, users are implicitly required to manually label a significant portion of these patches in the cross-validation setting before inferring the remaining patches. To mitigate the issue, in this study, we propose \toolname, the patch correctness assessment by adopting a large language model for code. Specifically, for patches generated by a new or unseen APR tool, \toolname does not need labeled patches of this new or unseen APR tool for training but directly queries the large language model for code to get predictions on the correctness labels without training. In this way, \toolname can reduce the manual labeling effort when building a model to automatically assess the correctness of generated patches of new APR tools. \toolname prioritizes labeled patches from existing APR tools that exhibit semantic similarity to those generated by new APR tools, enhancing the accuracy achieved by \toolname for patches from new APR tools. Our experimental results showed that \toolname can achieve an accuracy of 84.4% and an F1-score of 86.5% on average although no labeled patch of the new or unseen APR tool is available. In addition, our proposed technique outperformed the prior state-of-the-art by a large margin.

Software Engineering
Identifying Patch Correctness in Test-Based Program Repair

Yingfei Xiong,Xinyuan Liu,Muhan Zeng,Lu Zhang,Gang Huang

DOI: https://doi.org/10.1145/3180155.3180182

2018-07-27

Abstract:Test-based automatic program repair has attracted a lot of attention in recent years. However, the test suites in practice are often too weak to guarantee correctness and existing approaches often generate a large number of incorrect patches. To reduce the number of incorrect patches generated, we propose a novel approach that heuristically determines the correctness of the generated patches. The core idea is to exploit the behavior similarity of test case executions. The passing tests on original and patched programs are likely to behave similarly while the failing tests on original and patched programs are likely to behave differently. Also, if two tests exhibit similar runtime behavior, the two tests are likely to have the same test results. Based on these observations, we generate new test inputs to enhance the test suites and use their behavior similarity to determine patch correctness. Our approach is evaluated on a dataset consisting of 139 patches generated from existing program repair systems including jGenProg, Nopol, jKali, ACS and HDRepair. Our approach successfully prevented 56.3\% of the incorrect patches to be generated, without blocking any correct patches.

Software Engineering
A Large-Scale Empirical Review of Patch Correctness Checking Approaches

Jun Yang,Yuehan Wang,Yiling Lou,Ming Wen,Lingming Zhang

DOI: https://doi.org/10.1145/3611643.3616331

2023-01-01

Abstract:Automated Program Repair (APR) techniques have drawn wide attention from both academia and industry. Meanwhile, one main limitation with the current state-of-the-art APR tools is that patches passing all the original tests are not necessarily the correct ones wanted by developers, i.e., the plausible patch problem. To date, various Patch-Correctness Checking (PCC) techniques have been proposed to address this important issue. However, they are only evaluated on very limited datasets as the APR tools used for generating such patches can only explore a small subset of the search space of possible patches, posing serious threats to external validity to existing PCC studies. In this paper, we construct an extensive PCC dataset, PraPatch (the largest manually labeled PCC dataset to our knowledge), to revisit all nine state-of-the-art PCC techniques. More specifically, our PCC dataset PraPatch includes 1,988 patches generated from the recent PraPR APR tool, which leverages highly-optimized bytecode-level patch executions and can exhaustively explore all possible plausible patches within its large predefined search space (including well-known fixing patterns from various prior APR tools). Our extensive study of representative PCC techniques on PraPatch has revealed various findings, including: 1) the assumption made by existing static PCC techniques that correct patches are more similar to buggy code than incorrect plausible patches no longer holds, 2) state-of-the-art learning-based techniques tend to suffer from the dataset overfitting problem, 3) while dynamic techniques overall retain their effectiveness on our new dataset, their performance drops substantially on patches with more complicated changes and 4) the very recent naturalness-based techniques can substantially outperform traditional static techniques and could be a promising direction for PCC. Based on our findings, we also provide various guidelines/suggestions for advancing PCC in the near future.
Patch Correctness Assessment: A Survey

Zhiwei Fei,Jidong Ge,Chuanyi Li,Tianqi Wang,Yuning Li,Haodong Zhang,LiGuo Huang,Bin Luo

DOI: https://doi.org/10.1145/3702972

IF: 3.685

2024-01-01

ACM Transactions on Software Engineering and Methodology

Abstract:Most automated program repair methods rely on test cases to determine the correctness of the generated patches. However, due to the incompleteness of available test suites, some patches that pass all the test cases may still be incorrect. This issue is known as the patch overfitting problem. Overfitting problem is a longstanding problem in automated program repair. Due to overfitting patches, the patches obtained by automated program repair tools require further validation to determine their correctness. Researchers have proposed many methods to automatically assess the correctness of patches, but no systematic review provides a detailed introduction to this problem, the existing solutions, and the challenges. To address this deficiency, we systematically review the existing approaches to patch correctness assessment. We first offer a few examples of overfitting patches to acquire a more detailed understanding of this problem. We then propose a comprehensive categorization of publicly available techniques and datasets, examine the commonly used evaluation metrics, and perform an in-depth analysis of the effectiveness of the existing models in addressing the challenge of overfitting. Based on our analysis, we provided the difficulties encountered by current methodologies, alongside the possible avenues for future research exploration.
Crex: Predicting patch correctness in automated repair of C programs through transfer learning of execution semantics

Dapeng Yan,Kui Liu,Yuqing Niu,Li Li,Zhe Liu,Zhiming Liu,Jacques Klein,Tegawendé F. Bissyandé

DOI: https://doi.org/10.1016/j.infsof.2022.107043

IF: 3.9

2022-12-01

Information and Software Technology

Abstract:A significant body of automated program repair literature relies on test suites to assess the validity of generated patches. Because such oracles are weak, state-of-the-art repair tools can validate some patches that overfit the test cases but are actually incorrect. This situation has become a prime concern in APR, hindering its adoption by the industry. This work investigates execution semantic features based on micro-traces, a form of under-constrained dynamic traces. We build on transfer learning to explore function code representations that are amenable to semantic similarity computation and can therefore be leveraged for classifying patch correctness. Our Crex prototype implementation is based on the Trex framework. Experimental results on patches generated by the CoCoNut APR tool on CodeFlaws programs indicate that our approach can yield high accuracy in predicting patch correctness. The learned embeddings were proven to capture semantic similarities between functions, which was instrumental in training a classifier that identifies patch correctness by learning to discriminate between correctly patched code and incorrectly patched code based on their semantic similarity with the buggy function.

computer science, information systems, software engineering
Understanding Code Change with Micro-Changes

Lei Chen,Michele Lanza,Shinpei Hayashi

2024-09-16

Abstract:A crucial activity in software maintenance and evolution is the comprehension of the changes performed by developers, when they submit a pull request and/or perform a commit on the repository. Typically, code changes are represented in the form of code diffs, textual representations highlighting the differences between two file versions, depicting the added, removed, and changed lines. This simplistic representation must be interpreted by developers, and mentally lifted to a higher abstraction level, that more closely resembles natural language descriptions, and eases the creation of a mental model of the changes. However, the textual diff-based representation is cumbersome, and the lifting requires considerable domain knowledge and programming skills. We present an approach, based on the concept of micro-change, to overcome these difficulties, translating code diffs into a series of pre-defined change operations, which can be described in natural language. We present a catalog of micro-changes, together with an automated micro-change detector. To evaluate our approach, we performed an empirical study on a large set of open-source repositories, focusing on a subset of our micro-change catalog, namely those related to changes affecting the conditional logic. We found that our detector is capable of explaining more than 67% of the changes taking place in the systems under study.

Software Engineering
Automated Classification of Overfitting Patches with Statically Extracted Code Features

He Ye,Jian Gu,Matias Martinez,Thomas Durieux,Martin Monperrus

DOI: https://doi.org/10.1109/tse.2021.3071750

2021-08-06

Abstract:Automatic program repair (APR) aims to reduce the cost of manually fixing software defects. However, APR suffers from generating a multitude of overfitting patches, those patches that fail to correctly repair the defect beyond making the tests pass. This paper presents a novel overfitting patch detection system called ODS to assess the correctness of APR patches. ODS first statically compares a patched program and a buggy program in order to extract code features at the abstract syntax tree (AST) level. Then, ODS uses supervised learning with the captured code features and patch correctness labels to automatically learn a probabilistic model. The learned ODS model can then finally be applied to classify new and unseen program repair patches. We conduct a large-scale experiment to evaluate the effectiveness of ODS on patch correctness classification based on 10,302 patches from Defects4J, <a class="link-external link-http" href="http://Bugs.jar" rel="external noopener nofollow">this http URL</a> and Bears benchmarks. The empirical evaluation shows that ODS is able to correctly classify 71.9% of program repair patches from 26 projects, which improves the state-of-the-art. ODS is applicable in practice and can be employed as a post-processing procedure to classify the patches generated by different APR systems.

Software Engineering
APPT: Boosting Automated Patch Correctness Prediction via Fine-tuning Pre-trained Models

Quanjun Zhang,Chunrong Fang,Weisong Sun,Yan Liu,Tieke He,Xiaodong Hao,Zhenyu Chen

2024-01-16

Abstract:Automated program repair (APR) aims to fix software bugs automatically without human debugging efforts and plays a crucial role in software development and maintenance. Despite promising, APR is still challenged by a long-standing overfitting problem (i.e., the generated patch is plausible but overfitting). Various techniques have thus been proposed to address the overfitting problem. Recently, researchers have employed BERT to extract code features, which are then used to train a classifier for patch correctness prediction. However, BERT is restricted to feature extraction for classifier training without benefiting from the training process, potentially generating sub-optimal vector representations for patched code snippets. In this paper, we propose APPT, a pre-trained model-based automated patch correctness assessment technique by both pre-training and fine-tuning. APPT adopts a pre-trained model as the encoder stack, followed by an LSTM stack and a deep learning classifier. More importantly, the pre-trained model is fine-tuned in conjunction with other components as a whole pipeline to fully adapt it specifically for reasoning about patch correctness. We conduct an extensive experiment on 1,183 Defects4J patches and the experimental results show that APPT achieves prediction accuracy of 79.7% and recall of 83.2%, outperforming CACHE by 4.3% and 6.7%. Our additional investigation on 49,694 real-world patches shows that APPT achieves the optimum performance compared with existing representation learning techniques. We further investigate the impact of each component and find that they all positively contribute to APPT, e.g., the fine-tuning process and the LSTM stack increase F1-score by 10.22% and 4.11%, respectively. We also prove that adopting advanced pre-trained models can further provide substantial advancement, highlighting the generalizability of APPT.

Software Engineering
A Search Based Context-Aware Approach for Understanding and Localizing the Fault Via Weighted Call Graph

Jingxuan Tu,Xiaoyuan Xie,Yuming Zhou,Lin Chen,Baowen Xu

DOI: https://doi.org/10.1109/tsa.2016.20

2016-01-01

Abstract:Strictly speaking, fault localization includes assessing the code risk of being faulty and identifying the real fault. In practice, only highlighting some possible faulty statements is not helpful enough to reason the roots of the observed failures in a system. Programmers need to manually inspect the highlighted risky statements one by one, reading and understanding their contexts, in order to identify the real faulty ones. However, most related works have been focusing on risk assessment by simply ignoring the fault identification, which makes such techniques much less practical in real world. Therefore, in this paper, we propose a context-aware approach to assist fault comprehension and identification. Built on risk assessment results, our approach searches for the faults on Weighted Call Graph. In our approach the risky statements are re-ordered by function call chains, which can provide much richer information to understand the context and hence reduce the efforts in manual code inspection. Case studies with three open-source systems show that the proposed approach could help to improve the effectiveness of the whole fault localization process.
Attention: Not Just Another Dataset for Patch-Correctness Checking

Jun Yang,Yuehan Wang,Yiling Lou,Ming Wen,Lingming Zhang

DOI: https://doi.org/10.48550/arxiv.2207.06590

2023-01-01

Abstract: Automated Program Repair (APR) techniques have drawn wide attention from both academia and industry. Meanwhile, one main limitation with the current state-of-the-art APR tools is that patches passing all the original tests are not necessarily the correct ones wanted by developers, i.e., the plausible patch problem. To date, various Patch-Correctness Checking (PCC) techniques have been proposed to address this important issue. However, they are only evaluated on very limited datasets as the APR tools used for generating such patches can only explore a small subset of the search space of possible patches, posing serious threats to external validity to existing PCC studies. In this paper, we construct an extensive PCC dataset (the largest manually labeled PCC dataset to our knowledge) to revisit all state-of-the-art PCC techniques. More specifically, our PCC dataset includes 1,988 patches generated from the recent PraPR APR tool, which leverages highly-optimized bytecode-level patch executions and can exhaustively explore all possible plausible patches within its large predefined search space (including well-known fixing patterns from various prior APR tools). Our extensive study of representative PCC techniques on the new dataset has revealed various surprising findings and provided guidelines for future PCC research.

Context-Aware Code Change Embedding for Better Patch Correctness Assessment

TASK CONTEXT: A Tool for Predicting Code Context Models for Software Development Tasks

Context-Aware Patch Generation for Better Automated Program Repair

Fine-grained Commit-level Vulnerability Type Prediction by CWE Tree Structure.

Patchmatch: A Tool for Locating Patches of Open Source Project Vulnerabilities

Just-In-Time Defect Identification and Localization: A Two-Phase Framework.

The Inversive Relationship Between Bugs and Patches: An Empirical Study

Neural Program Repair with Program Dependence Analysis and Effective Filter Mechanism

Patch Space Exploration using Static Analysis Feedback

An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation

Shaping Program Repair Space with Existing Patches and Similar Code

PatchZero: Zero-Shot Automatic Patch Correctness Assessment

Identifying Patch Correctness in Test-Based Program Repair

A Large-Scale Empirical Review of Patch Correctness Checking Approaches

Patch Correctness Assessment: A Survey

Crex: Predicting patch correctness in automated repair of C programs through transfer learning of execution semantics

Understanding Code Change with Micro-Changes

Automated Classification of Overfitting Patches with Statically Extracted Code Features

APPT: Boosting Automated Patch Correctness Prediction via Fine-tuning Pre-trained Models

A Search Based Context-Aware Approach for Understanding and Localizing the Fault Via Weighted Call Graph

Attention: Not Just Another Dataset for Patch-Correctness Checking