Abstract:Compared to traditional RNN-based models, abstractive summarization systems based on Pre-trained Language Models (PTMs) achieve dramatic improvements in readability. Thus, in the field of abstractive summarization, more attention should be devoted to the faithfulness issue that predicted summaries are not factually consistent with source texts. To alleviate this disadvantage, we propose a novel Syntax-Enriched Abstractive Summarization (SEASum) framework, which utilizes graph attention networks (GATs) to introduce syntactic features of source texts to generate faithful summaries. In the SEASum framework, the PTM-based semantic encoder encodes word sequence, while the GAT-based syntactic encoder captures explicit syntax, i.e., part-of-speech tags, parse trees, and dependency-based relative positions of source documents. A feature fusion module is introduced to incorporate encoded syntactic features into the summarization framework. Based on the proposed SEASum framework, we develop two summarization models: 1) parallel SEASum model, in which the semantic encoder and syntactic encoder work in parallel, a multi-head attention module fused two-stream features for the following decoding process; 2) cascaded SEASum model, which takes contextual word embeddings from semantic encoder as node embeddings for the syntactic encoder and employs highway networks to regulate information flow. Experimental results on CNN/DailyMail and Reddit-TIFU (short) datasets show our parallel SEASum model and cascaded SEASum model outperform state-of-the-art abstractive summarization approaches in the faithfulness measurement. The results also demonstrate that cascaded SEASum model is more effective than parallel SEASum model in boosting faithfulness.

Modeling Source Syntax and Semantics for Neural AMR Parsing

Neural amr: Sequence-to-sequence models for parsing and generation

Modeling Source Syntax for Neural Machine Translation

Improving Neural Machine Translation with AMR Semantic Graphs

Exploration and Comparison of Diverse Approaches for Integrating Syntactic Knowledge into AMR Parsing

Integrating Syntactic and Semantic Knowledge in AMR Parsing with Heterogeneous Graph Attention Network

Improving AMR Parsing with Sequence-to-Sequence Pre-training

From text to graph: a general transition-based AMR parsing using neural network

AMR Parsing as Sequence-to-Graph Transduction

Improving AMR Parsing by Exploiting the Dependency Parsing As an Auxiliary Task.

AMR Parsing with Latent Structural Information.

AMR Parsing Via Graph-Sequence Iterative Inference

Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations

Syntax-aware Neural Semantic Role Labeling

SEASum: Syntax-Enriched Abstractive Summarization

Syntax-Aware Complex-Valued Neural Machine Translation

Parsing All: Syntax and Semantics, Dependencies and Spans

Modeling Graph Structure in Transformer for Better AMR-to-Text Generation.

Semantic Neural Machine Translation Using AMR

Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing

Logical Parsing from Natural Language Based on a Neural Translation Model.