Abstract:Compared to traditional RNN-based models, abstractive summarization systems based on Pre-trained Language Models (PTMs) achieve dramatic improvements in readability. Thus, in the field of abstractive summarization, more attention should be devoted to the faithfulness issue that predicted summaries are not factually consistent with source texts. To alleviate this disadvantage, we propose a novel Syntax-Enriched Abstractive Summarization (SEASum) framework, which utilizes graph attention networks (GATs) to introduce syntactic features of source texts to generate faithful summaries. In the SEASum framework, the PTM-based semantic encoder encodes word sequence, while the GAT-based syntactic encoder captures explicit syntax, i.e., part-of-speech tags, parse trees, and dependency-based relative positions of source documents. A feature fusion module is introduced to incorporate encoded syntactic features into the summarization framework. Based on the proposed SEASum framework, we develop two summarization models: 1) parallel SEASum model, in which the semantic encoder and syntactic encoder work in parallel, a multi-head attention module fused two-stream features for the following decoding process; 2) cascaded SEASum model, which takes contextual word embeddings from semantic encoder as node embeddings for the syntactic encoder and employs highway networks to regulate information flow. Experimental results on CNN/DailyMail and Reddit-TIFU (short) datasets show our parallel SEASum model and cascaded SEASum model outperform state-of-the-art abstractive summarization approaches in the faithfulness measurement. The results also demonstrate that cascaded SEASum model is more effective than parallel SEASum model in boosting faithfulness.

Title Generation with Quasi-Synchronous Grammar

SEASum: Syntax-Enriched Abstractive Summarization

Learning Structural Information for Syntax-Controlled Paraphrase Generation.

Improving Generation Diversity Via Syntax-Controlled Paraphrasing

Bridging Visual and Textual Semantics: Towards Consistency for Unbiased Scene Graph Generation

A Lightweight Constrained Generation Alternative for Query-focused Summarization

A Syntax-Augmented and Headline-Aware Neural Text Summarization Method

Joint Parsing and Generation for Abstractive Summarization

Augmenting Paraphrase Generation with Syntax Information Using Graph Convolutional Networks.

SGG: Learning to Select, Guide, and Generate for Keyphrase Generation

Summarization With Self-Aware Context Selecting Mechanism

A New Approach to Overgenerating and Scoring Abstractive Summaries

A SP synchronous generation model for statistical machine translation 1

Explicit Syntactic Guidance for Neural Text Generation

Syntax-guided Controlled Generation of Paraphrases

A Submodular Optimization-Based VAE-Transformer Framework for Paraphrase Generation.

Dependency-based N-Gram Models for General Purpose Sentence Realisation

Extractive-abstractive Summarization with Pointer and Coverage Mechanism

Document-level paraphrase generation base on attention enhanced graph LSTM

A Unified Framework for Abstractive Summarization over Prompt Language Model and Pointer Mechanism.

Syntax-guided Text Generation Via Graph Neural Network