Abstract:Compared to traditional RNN-based models, abstractive summarization systems based on Pre-trained Language Models (PTMs) achieve dramatic improvements in readability. Thus, in the field of abstractive summarization, more attention should be devoted to the faithfulness issue that predicted summaries are not factually consistent with source texts. To alleviate this disadvantage, we propose a novel Syntax-Enriched Abstractive Summarization (SEASum) framework, which utilizes graph attention networks (GATs) to introduce syntactic features of source texts to generate faithful summaries. In the SEASum framework, the PTM-based semantic encoder encodes word sequence, while the GAT-based syntactic encoder captures explicit syntax, i.e., part-of-speech tags, parse trees, and dependency-based relative positions of source documents. A feature fusion module is introduced to incorporate encoded syntactic features into the summarization framework. Based on the proposed SEASum framework, we develop two summarization models: 1) parallel SEASum model, in which the semantic encoder and syntactic encoder work in parallel, a multi-head attention module fused two-stream features for the following decoding process; 2) cascaded SEASum model, which takes contextual word embeddings from semantic encoder as node embeddings for the syntactic encoder and employs highway networks to regulate information flow. Experimental results on CNN/DailyMail and Reddit-TIFU (short) datasets show our parallel SEASum model and cascaded SEASum model outperform state-of-the-art abstractive summarization approaches in the faithfulness measurement. The results also demonstrate that cascaded SEASum model is more effective than parallel SEASum model in boosting faithfulness.

Abstract Meaning Representation for Multi-Document Summarization

Text Summarization using Abstract Meaning Representation

Unsupervised Abstractive Summarization Via Sentence Rewriting

SEASum: Syntax-Enriched Abstractive Summarization

Entity-Aware Abstractive Multi-Document Summarization.

Multi-Document Summarization Based On Two-Level Sparse Representation Model

Augmented Abstractive Summarization with Document-Level Semantic Graph

Abstractive Multi-Document Summarization Based on Semantic Link Network

GATSum: Graph-Based Topic-Aware Abstract Text Summarization

Topic-Aware Abstractive Text Summarization

Leveraging Graph to Improve Abstractive Multi-Document Summarization.

Learning to Consider Relevance and Redundancy Dynamically for Abstractive Multi-document Summarization

Joint Parsing and Generation for Abstractive Summarization

Topic-Centric Unsupervised Multi-Document Summarization of Scientific and News Articles

Modeling Endorsement for Multi-Document Abstractive Summarization

Abstractive summarization incorporating graph knowledge

Large-Scale Multi-Document Summarization with Information Extraction and Compression

Multi-Document Abstractive Summarization Using Chunk-graph and Recurrent Neural Network

DocSum: Domain-Adaptive Pre-training for Document Abstractive Summarization

An Effective Joint Framework for Document Summarization

Automatic Multi-Document Summarization for Digital Libraries