Dialog Summarization for Software Collaborative Platform Via Tuning Pre-Trained Models
Guodong Fan,Shizhan Chen,Hongyue Wu,Cuiyun Gao,Jianmao Xiao,Xiao Xue,Zhiyong Feng
DOI: https://doi.org/10.1016/j.jss.2023.111763
IF: 3.5
2023-01-01
Journal of Systems and Software
Abstract:Software collaborative platforms, e.g., Gitter live chat and GitHub Discussions, are essential in software maintenance. Summarizing the live chat logs is useful for extracting, retrieving, and sharing knowledge for software developers. Automatic text summarization has been studied in many areas such as code summarization, and title generation. However, the previous studies rely on rich collected labeled data for model training which are absent for the noisy interleaved dialogs, resulting in poor performance in the few-shot scenario. To tackle the issue, we propose a novel Automatic Dialog Summarization Approach based on pre-trained models, named ADSum. To alleviate the high-cost problem of the from-scratch manual annotation, ADSum finetunes the Text-To-Text Transfer Transformer (T5) model by exploiting the discussion posts on GitHub, and then recommends summaries for an annotator. To solve the poor performance in the few-shot scenario, we propose to employ the prompt tuning paradigm for tuning the T5 model by exploiting the disentangled dialog data on Gitter. Meanwhile, the soft prompt is used to avoid the manual effort of designing appropriate prompt templates. To verify the effectiveness of our approach, we extract 38,964 high-quality discussion posts from GitHub and manually annotate 3,039 dialog summarizations from Gitter. Experimental results show our approach achieves state-of-the-art performance in terms of three performance metrics. In particular, our proposed method outperformed the Transformer-based and other pre-training models by 39% and 14%, respectively, on the GitHub dataset regarding the Rouge-L metric. The experiments of handling data scarcity and a human evaluation also confirm the effectiveness of ADSum.