Analyzing Sentence Fusion in Abstractive Summarization

Logan Lebanoff,John Muchovej,Franck Dernoncourt,Doo Soon Kim,Seokhwan Kim,Walter Chang,Fei Liu

DOI: https://doi.org/10.48550/arXiv.1910.00203

2019-10-01

Abstract:While recent work in abstractive summarization has resulted in higher scores in automatic metrics, there is little understanding on how these systems combine information taken from multiple document sentences. In this paper, we analyze the outputs of five state-of-the-art abstractive summarizers, focusing on summary sentences that are formed by sentence fusion. We ask assessors to judge the grammaticality, faithfulness, and method of fusion for summary sentences. Our analysis reveals that system sentences are mostly grammatical, but often fail to remain faithful to the original article.

Computation and Language

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in abstractive summarization, how can the system effectively integrate information from sentences in multiple documents while maintaining grammatical correctness and fidelity to the original text. Although high scores have been achieved in automatic evaluation metrics in recent years, these systems still have deficiencies in their performance when merging multi - sentence information, especially in terms of grammatical correctness and information authenticity. The paper explores these issues by analyzing the outputs of five state - of - the - art abstractive text summarization systems, especially those summary sentences generated through sentence fusion. The researchers focus on the grammatical correctness, fidelity of the summary sentences and the fusion methods, and conduct in - depth analysis of these aspects through manual evaluation. This provides valuable insights for improving current summary systems, especially for the key task of sentence fusion.