Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization

Beliz Gunel,Chenguang Zhu,Michael Zeng,Xuedong Huang

DOI: https://doi.org/10.48550/arXiv.2006.15435

2020-06-28

Abstract:Neural models have become successful at producing abstractive summaries that are human-readable and fluent. However, these models have two critical shortcomings: they often don't respect the facts that are either included in the source article or are known to humans as commonsense knowledge, and they don't produce coherent summaries when the source article is long. In this work, we propose a novel architecture that extends Transformer encoder-decoder architecture in order to improve on these shortcomings. First, we incorporate entity-level knowledge from the Wikidata knowledge graph into the encoder-decoder architecture. Injecting structural world knowledge from Wikidata helps our abstractive summarization model to be more fact-aware. Second, we utilize the ideas used in Transformer-XL language model in our proposed encoder-decoder architecture. This helps our model with producing coherent summaries even when the source article is long. We test our model on CNN/Daily Mail summarization dataset and show improvements on ROUGE scores over the baseline Transformer model. We also include model predictions for which our model accurately conveys the facts, while the baseline Transformer model doesn't.

Computation and Language

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on two aspects:

1. **Fact - respect**: Existing neural abstractive summarization models often do not respect the facts or human common - sense knowledge contained in the source article when generating summaries. This means that the generated summaries may contain incorrect information, which is a serious problem in applications requiring high accuracy.

2. **Coherence of long documents**: When the source article is long, the summaries generated by existing models often lack coherence. This makes the summaries difficult to understand, especially when dealing with articles on complex topics.

To solve these problems, the author proposes a new architecture that extends the Transformer encoder - decoder structure in the following ways:

- **Introducing entity - level knowledge**: By injecting entity - level knowledge from the Wikidata knowledge graph into the encoder - decoder architecture, the model becomes more fact - sensitive.
- **Utilizing the idea of Transformer - XL**: Drawing on the idea of the Transformer - XL language model to handle long - term dependencies in long documents, thereby generating more coherent summaries.

Through these improvements, the author hopes to improve the factual accuracy and coherence of the summaries while maintaining the fluency of the summaries. Experimental results show that the model has an improved ROUGE score on the CNN/Daily Mail dataset, and the performance is particularly significant on the test subset with high entity density.