EDU-level Extractive Summarization with Varying Summary Lengths

Yuping Wu,Ching-Hsun Tseng,Jiayu Shang,Shengzhong Mao,Goran Nenadic,Xiao-Jun Zeng

DOI: https://doi.org/10.48550/arXiv.2210.04029

2023-03-13

Abstract:Extractive models usually formulate text summarization as extracting fixed top-$k$ salient sentences from the document as a summary. Few works exploited extracting finer-grained Elementary Discourse Unit (EDU) with little analysis and justification for the extractive unit selection. Further, the selection strategy of the fixed top-$k$ salient sentences fits the summarization need poorly, as the number of salient sentences in different documents varies and therefore a common or best $k$ does not exist in reality. To fill these gaps, this paper first conducts the comparison analysis of oracle summaries based on EDUs and sentences, which provides evidence from both theoretical and experimental perspectives to justify and quantify that EDUs make summaries with higher automatic evaluation scores than sentences. Then, considering this merit of EDUs, this paper further proposes an EDU-level extractive model with Varying summary Lengths and develops the corresponding learning algorithm. EDU-VL learns to encode and predict probabilities of EDUs in the document, generate multiple candidate summaries with varying lengths based on various $k$ values, and encode and score candidate summaries, in an end-to-end training manner. Finally, EDU-VL is experimented on single and multi-document benchmark datasets and shows improved performances on ROUGE scores in comparison with state-of-the-art extractive models, and further human evaluation suggests that EDU-constituent summaries maintain good grammaticality and readability.

Computation and Language

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are two key issues in the existing extractive summarization methods:

1. **Selection of extraction units**: Most of the existing extractive summarization methods mainly focus on extracting sentences from documents as summaries, and few studies have explored the use of more fine - grained text fragments (such as Elementary Discourse Unit, EDU) as extraction units. The paper points out that a sentence may contain multiple clauses, and some of these clauses may be unimportant, which may lead to redundant or inaccurate generated summaries. Therefore, the paper proposes to use EDU as an extraction unit and verifies the superiority of EDU in generating summaries through theoretical analysis and experiments.

2. **Flexibility of summary length**: Existing extractive summarization methods usually use the fixed top - k most significant sentences to generate summaries. This method ignores the differences in the number of significant sentences in different documents, resulting in insufficient flexibility in the length of the generated summaries. The paper proposes a method that allows the summary length to vary. It generates candidate summaries of different lengths by adjusting the value of k and selects the optimal summary length.

To solve the above problems, the paper has carried out the following work:

- **Theoretical and experimental analysis**: Through theoretical proof and experimental verification, it is shown that using EDU as an extraction unit can obtain higher automatic evaluation scores (such as ROUGE scores) compared to using sentences. Specifically, the paper defines "Oracle Summary" at the EDU and sentence levels and proves the advantage of EDU in generating high - quality summaries by comparing the ROUGE scores of these summaries.

- **Model and algorithm development**: A proposed EDU - based extractive summarization model (EDU - VL) can generate multiple candidate summaries of different lengths according to different k values after encoding the EDU in the document and select the final summary by calculating the similarity between the candidate summaries and the original document. This process is carried out in an end - to - end training framework, ensuring the effectiveness and robustness of the model.

- **Experimental verification**: Experiments were carried out on multiple benchmark datasets, including single - document summarization datasets (such as CNN/DailyMail, XSum, Reddit, WikiHow) and multi - document summarization datasets (such as Multi - News). The experimental results show that EDU - VL is superior to the existing state - of - the - art extractive summarization models in ROUGE scores, and human evaluation also shows that the summaries composed of EDU have good grammatical structures and readability.

In conclusion, through theoretical analysis and empirical research, this paper proves the superiority of using EDU as an extraction unit and proposes a new method that can generate high - quality summaries with flexible lengths.