Abstract:Automatic text summarization aims at condensing a document to a shorter version while preserving the key information. Different from extractive summarization which simply selects text fragments from the document, abstractive summarization generates the summary in a word-by-word manner. Most current state-of-the-art (SOTA) abstractive summarization methods are based on the Transformer-based encoder-decoder architecture and focus on novel self-supervised objectives in pre-training. While these models well capture the contextual information among words in documents, little attention has been paid to incorporating global semantics to better fine-tune for the downstream abstractive summarization task. In this study, we propose a topic-aware abstractive summarization (TAAS) framework by leveraging the underlying semantic structure of documents represented by their latent topics. Specifically, TAAS seamlessly incorporates a neural topic modeling into an encoder-decoder based sequence generation procedure via attention for summarization. This design is able to learn and preserve global semantics of documents and thus makes summarization effective, which has been proved by our experiments on real-world datasets. As compared to several cutting-edge baseline methods, we show that TAAS outperforms BART, a well-recognized SOTA model, by 2%, 8%, and 12% regarding the F measure of ROUGE-1, ROUGE-2, and ROUGE-L, respectively. TAAS also achieves comparable performance to PEGASUS and ProphetNet, which is difficult to accomplish given that training PEGASUS and ProphetNet requires enormous computing capacity beyond what we used in this study.

Utterance-Level Latent Topic Transition Modeling for Spoken Documents and Its Application in Automatic Summarization

Unsupervised Topic Modeling Approaches to Decision Summarization in Spoken Meetings

Supervised Spoken Document Summarization Based On Structured Support Vector Machine With Utterance Clusters As Hidden Variables

A Latent Topic Model for Linked Documents

Using Topic Labels for Text Summarization.

Multi-Document Summarization Using Sentence-based Topic Models.

A Topic Model for Linked Documents and Update Rules for Its Estimation

Automatic Labeling of Topic Models Using Text Summaries

Investigating the Impact of Text Summarization on Topic Modeling

A Neural Abstractive Summarization Model Guided with Topic Sentences.

Topic-Aware Abstractive Text Summarization

Supervised Spoken Document Summarization Jointly Considering Utterance Importance and Redundancy by Structured Support Vector Machine.

Refine Bigram PLSA Model by Assigning Latent Topics Unevenly

Subtopic-Based Multimodality Ranking for Topic-Focused Multidocument Summarization.

Semantic Query Expansion and Context-Based Discriminative Term Modeling for Spoken Document Retrieval

A new latent semantic analysis language model

Topic-Guided Abstractive Text Summarization: a Joint Learning Approach

Topic Modeling Based Text Summarization Approach

Tipster: A Topic-Guided Language Model for Topic-Aware Text Segmentation.

Heterogeneous-Length Text Topic Modeling for Reader-Aware Multi-Document Summarization.

A Latent Topic Model with Markov Transition for Process Data