Abstract:Compared to traditional RNN-based models, abstractive summarization systems based on Pre-trained Language Models (PTMs) achieve dramatic improvements in readability. Thus, in the field of abstractive summarization, more attention should be devoted to the faithfulness issue that predicted summaries are not factually consistent with source texts. To alleviate this disadvantage, we propose a novel Syntax-Enriched Abstractive Summarization (SEASum) framework, which utilizes graph attention networks (GATs) to introduce syntactic features of source texts to generate faithful summaries. In the SEASum framework, the PTM-based semantic encoder encodes word sequence, while the GAT-based syntactic encoder captures explicit syntax, i.e., part-of-speech tags, parse trees, and dependency-based relative positions of source documents. A feature fusion module is introduced to incorporate encoded syntactic features into the summarization framework. Based on the proposed SEASum framework, we develop two summarization models: 1) parallel SEASum model, in which the semantic encoder and syntactic encoder work in parallel, a multi-head attention module fused two-stream features for the following decoding process; 2) cascaded SEASum model, which takes contextual word embeddings from semantic encoder as node embeddings for the syntactic encoder and employs highway networks to regulate information flow. Experimental results on CNN/DailyMail and Reddit-TIFU (short) datasets show our parallel SEASum model and cascaded SEASum model outperform state-of-the-art abstractive summarization approaches in the faithfulness measurement. The results also demonstrate that cascaded SEASum model is more effective than parallel SEASum model in boosting faithfulness.

A Unified Summarization Model with Semantic Guide and Keyword Coverage Mechanism

An Abstractive Summarization Model Based on Joint-Attention Mechanism and a Priori Knowledge

A Syntax-Augmented and Headline-Aware Neural Text Summarization Method

SEASum: Syntax-Enriched Abstractive Summarization

An Abstractive Summarizer Based on Improved Pointer-Generator Network

Abstract Summarization Model Based on Semantic Graphs and Entity Pointers

HITS-based attentional neural model for abstractive summarization

Incorporating word attention with convolutional neural networks for abstractive summarization

Topic-Aware Encoding for Extractive Summarization

Abstractive Summarization Improved by WordNet-based Extractive Sentences

A Multi-View Abstractive Summarization Model Jointly Considering Semantics and Sentiment

A Simple Semantics and Topic-aware Method to Enhance Abstractive Summarization

Keywords-Based Auxiliary Information Network for Abstractive Summarization

SemSUM: Semantic Dependency Guided Neural Abstractive Summarization

Abstractive Summarization with Keyword and Generated Word Attention.

A Unified Framework for Abstractive Summarization over Prompt Language Model and Pointer Mechanism.

Neural Abstractive Summarization: A Brief Survey

Extractive-abstractive Summarization with Pointer and Coverage Mechanism

A Sequence-to-Sequence Text Summarization Model with Topic Based Attention Mechanism.

A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss

Abstractive Document Summarization via Neural Model with Joint Attention