Abstract:Compared to traditional RNN-based models, abstractive summarization systems based on Pre-trained Language Models (PTMs) achieve dramatic improvements in readability. Thus, in the field of abstractive summarization, more attention should be devoted to the faithfulness issue that predicted summaries are not factually consistent with source texts. To alleviate this disadvantage, we propose a novel Syntax-Enriched Abstractive Summarization (SEASum) framework, which utilizes graph attention networks (GATs) to introduce syntactic features of source texts to generate faithful summaries. In the SEASum framework, the PTM-based semantic encoder encodes word sequence, while the GAT-based syntactic encoder captures explicit syntax, i.e., part-of-speech tags, parse trees, and dependency-based relative positions of source documents. A feature fusion module is introduced to incorporate encoded syntactic features into the summarization framework. Based on the proposed SEASum framework, we develop two summarization models: 1) parallel SEASum model, in which the semantic encoder and syntactic encoder work in parallel, a multi-head attention module fused two-stream features for the following decoding process; 2) cascaded SEASum model, which takes contextual word embeddings from semantic encoder as node embeddings for the syntactic encoder and employs highway networks to regulate information flow. Experimental results on CNN/DailyMail and Reddit-TIFU (short) datasets show our parallel SEASum model and cascaded SEASum model outperform state-of-the-art abstractive summarization approaches in the faithfulness measurement. The results also demonstrate that cascaded SEASum model is more effective than parallel SEASum model in boosting faithfulness.

A Hierarchical Attention Seq2seq Model with CopyNet for Text Summarization

A Hierarchical Neural Abstractive Summarization with Self-Attention Mechanism

A Hierarchical Hybrid Neural Network Architecture for Chinese Text Summarization.

SEASum: Syntax-Enriched Abstractive Summarization

Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond

Abstract Text Summarization with a Convolutional Seq2seq Model

Seq2seq Model with RNN Attention for Abstractive Summarization

A Syntax-Augmented and Headline-Aware Neural Text Summarization Method

Abstractive text summarization model combining a hierarchical attention mechanism and multiobjective reinforcement learning

Hybrid Non-Local Network for Abstractive Summarization

Hierarchical Heterogeneous Graph Attention Network for Syntax-Aware Summarization

Abstractive Document Summarization via Neural Model with Joint Attention

Summary-aware Attention for Social Media Short Text Abstractive Summarization

Extractive-abstractive Summarization with Pointer and Coverage Mechanism

Abstractive Summarization Improved by WordNet-based Extractive Sentences

Reading More Efficiently: Multi-Sentence Summarization With A Dual Attention And Copy-Generator Network

HITS-based attentional neural model for abstractive summarization

Selective and Coverage Multi-head Attention for Abstractive Summarization

Abstractive Document Summarization with a Graph-Based Attentional Neural Model.

Incorporating word attention with convolutional neural networks for abstractive summarization

Gated Graph Neural Attention Networks for abstractive summarization