Abstract:Compared to traditional RNN-based models, abstractive summarization systems based on Pre-trained Language Models (PTMs) achieve dramatic improvements in readability. Thus, in the field of abstractive summarization, more attention should be devoted to the faithfulness issue that predicted summaries are not factually consistent with source texts. To alleviate this disadvantage, we propose a novel Syntax-Enriched Abstractive Summarization (SEASum) framework, which utilizes graph attention networks (GATs) to introduce syntactic features of source texts to generate faithful summaries. In the SEASum framework, the PTM-based semantic encoder encodes word sequence, while the GAT-based syntactic encoder captures explicit syntax, i.e., part-of-speech tags, parse trees, and dependency-based relative positions of source documents. A feature fusion module is introduced to incorporate encoded syntactic features into the summarization framework. Based on the proposed SEASum framework, we develop two summarization models: 1) parallel SEASum model, in which the semantic encoder and syntactic encoder work in parallel, a multi-head attention module fused two-stream features for the following decoding process; 2) cascaded SEASum model, which takes contextual word embeddings from semantic encoder as node embeddings for the syntactic encoder and employs highway networks to regulate information flow. Experimental results on CNN/DailyMail and Reddit-TIFU (short) datasets show our parallel SEASum model and cascaded SEASum model outperform state-of-the-art abstractive summarization approaches in the faithfulness measurement. The results also demonstrate that cascaded SEASum model is more effective than parallel SEASum model in boosting faithfulness.

Improving Semantic Relevance For Sequence-To-Sequence Learning Of Chinese Social Media Text Summarization

A Semantic Relevance Based Neural Network for Text Summarization and Text Simplification

Text Semantics Based Automatic Summarization for Chinese Videos

Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarization

Sequence Generative Adversarial Network for Chinese Social Media Text Summarization

Summary-aware Attention for Social Media Short Text Abstractive Summarization

Regularizing Output Distribution of Abstractive Chinese Social Media Text Summarization for Improved Semantic Consistency

A Syntax-Augmented and Headline-Aware Neural Text Summarization Method

SemSUM: Semantic Dependency Guided Neural Abstractive Summarization

A Multi-View Abstractive Summarization Model Jointly Considering Semantics and Sentiment

A Hierarchical Hybrid Neural Network Architecture for Chinese Text Summarization.

SEASum: Syntax-Enriched Abstractive Summarization

A Unified Summarization Model with Semantic Guide and Keyword Coverage Mechanism

Improving Social Media Text Summarization by Learning Sentence Weight Distribution

Text Summarization Generation Based on Semantic Similarity

Social Media Opinion Summarization Using Emotion Cognition and Convolutional Neural Networks

Improved BIO-based Chinese Automatic Abstract-generation Model

Abstractive Social Media Text Summarization Using Selective Reinforced Seq2Seq Attention Model.

Summarization With Self-Aware Context Selecting Mechanism

Deep Learning Approaches to Semantic Relevance Modeling for Chinese Question-Answer Pairs.