TitleGen-FL: Quality prediction-based filter for automated issue title generation

Hao Lin,Xiang Chen,Xuejiao Chen,Zhanqi Cui,Yun Miao,Shan Zhou,Jianmin Wang,Zhan Su

DOI: https://doi.org/10.1016/j.jss.2022.111513

IF: 3.5

2023-01-01

Journal of Systems and Software

Abstract:To automatically generate the issue title, researchers formulated the problem into a one-sentence summarization problem and then proposed an effective method iTAPE. However, after analyzing the quality of the titles generated by the method iTAPE, which is measured by ROUGE-L, we find the ROUGE-L scores of only 42.7% of titles can exceed 0.3. This means the quality of the generated titles is not satisfactory and can limit the practicability of the method iTAPE. Therefore, we propose a quality prediction-based filter TitleGen-FL, which can predict whether the method iTAPE can generate a high-quality title after analyzing the issue body. To achieve this goal, TitleGen-FL includes the deep learning-based (DL) module and the information retrieval-based (IR) module. If either of these two modules predict that the high-quality title cannot be generated by iTAPE, TitleGen-FL can automatically filter this issue and return a warning message. To evaluate the effectiveness of our proposed filter, we select the benchmark dataset gathered from real-world open-source projects as our experimental subject. Both automatic evaluation and human study show that our proposed filter TitleGen-FL can effectively filter the issues, which cannot generate high-quality titles by iTAPE.

computer science, theory & methods, software engineering