Abstract:Compared to traditional RNN-based models, abstractive summarization systems based on Pre-trained Language Models (PTMs) achieve dramatic improvements in readability. Thus, in the field of abstractive summarization, more attention should be devoted to the faithfulness issue that predicted summaries are not factually consistent with source texts. To alleviate this disadvantage, we propose a novel Syntax-Enriched Abstractive Summarization (SEASum) framework, which utilizes graph attention networks (GATs) to introduce syntactic features of source texts to generate faithful summaries. In the SEASum framework, the PTM-based semantic encoder encodes word sequence, while the GAT-based syntactic encoder captures explicit syntax, i.e., part-of-speech tags, parse trees, and dependency-based relative positions of source documents. A feature fusion module is introduced to incorporate encoded syntactic features into the summarization framework. Based on the proposed SEASum framework, we develop two summarization models: 1) parallel SEASum model, in which the semantic encoder and syntactic encoder work in parallel, a multi-head attention module fused two-stream features for the following decoding process; 2) cascaded SEASum model, which takes contextual word embeddings from semantic encoder as node embeddings for the syntactic encoder and employs highway networks to regulate information flow. Experimental results on CNN/DailyMail and Reddit-TIFU (short) datasets show our parallel SEASum model and cascaded SEASum model outperform state-of-the-art abstractive summarization approaches in the faithfulness measurement. The results also demonstrate that cascaded SEASum model is more effective than parallel SEASum model in boosting faithfulness.

Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries

Hierarchical Summary to Article Generation

Long text outline generation: Chinese text outline based on unsupervised framework and large language mode

Abstractive text summarization model combining a hierarchical attention mechanism and multiobjective reinforcement learning

A study of extractive summarization of long documents incorporating local topic and hierarchical information

Write Summary Step-by-Step: A Pilot Study of Stepwise Summarization

GoSum: Extractive Summarization of Long Documents by Reinforcement Learning and Graph Organized discourse state

A New Approach to Overgenerating and Scoring Abstractive Summaries

Unsupervised Abstractive Summarization Via Sentence Rewriting

Generating Multiple-Length Summaries via Reinforcement Learning for Unsupervised Sentence Summarization

SEASum: Syntax-Enriched Abstractive Summarization

Hierarchical Text Generation using an Outline

A Two-Stage Long Text Summarization Method Based on Discourse Structure

EDU-level Extractive Summarization with Varying Summary Lengths

Topic-Guided Abstractive Text Summarization: a Joint Learning Approach

A Hierarchical Hybrid Neural Network Architecture for Chinese Text Summarization.

Generating Long-form Story Using Dynamic Hierarchical Outlining with Memory-Enhancement

Abstract Text Summarization with a Convolutional Seq2seq Model

Integrating Extractive and Abstractive Models for Long Text Summarization

An Unsupervised Extractive Summarization Method Based on Multi-Round Computation

Hie-Transformer: A Hierarchical Hybrid Transformer For Abstractive Article Summarization