Abstract:In the era of social networks, the rapid growth of data mining in information retrieval and natural language processing makes automatic text summarization necessary. Currently, pretrained word embedding and sequence to sequence models can be effectively adapted in social network summarization to extract significant information with strong encoding capability. However, how to tackle the long text dependence and utilize the latent topic mapping has become an increasingly crucial challenge for these models. In this article, we propose a topic-aware extractive and abstractive summarization model named T-BERTSum, based on Bidirectional Encoder Representations from Transformers (BERTs). This is an improvement over previous models, in which the proposed approach can simultaneously infer topics and generate summarization from social texts. First, the encoded latent topic representation, through the neural topic model (NTM), is matched with the embedded representation of BERT, to guide the generation with the topic. Second, the long-term dependencies are learned through the transformer network to jointly explore topic inference and text summarization in an end-to-end manner. Third, the long short-term memory (LSTM) network layers are stacked on the extractive model to capture sequence timing information, and the effective information is further filtered on the abstractive model through a gated network. In addition, a two-stage extractive–abstractive model is constructed to share the information. Compared with the previous work, the proposed model T-BERTSum focuses on pretrained external knowledge and topic mining to capture more accurate contextual representations. Experimental results on the CNN/Daily mail and XSum datasets demonstrate that our proposed model achieves new state-of-the-art results while generating consistent topics compared with the most advanced method.

A Text Abstraction Summary Model Based on BERT Word Embedding and Reinforcement Learning

Abstractive text summarization model combining a hierarchical attention mechanism and multiobjective reinforcement learning

T-BERTSum: Topic-Aware Text Summarization Based on BERT

An Extraction-Abstraction Hybrid Approach for Long Document Summarization

Text Summarization with Pretrained Encoders

Abstractive Summarization Improved by WordNet-based Extractive Sentences

A Reinforced Improved Attention Model for Abstractive Text Summarization

An Abstractive Summarization Model Based on Joint-Attention Mechanism and a Priori Knowledge

Unified extractive-abstractive summarization: a hybrid approach utilizing BERT and transformer models for enhanced document summarization

Unsupervised Abstractive Summarization Via Sentence Rewriting

Hie-Transformer: A Hierarchical Hybrid Transformer For Abstractive Article Summarization

Inverse Reinforcement Learning for Text Summarization

Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting

Exploring Human-Like Reading Strategy for Abstractive Text Summarization

A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss

Extractive-Abstractive: A Two-Stage Model for Long Text Summarization

Generating Multiple-Length Summaries via Reinforcement Learning for Unsupervised Sentence Summarization

Abstract Text Summarization with a Convolutional Seq2seq Model

Improving the readability and saliency of abstractive text summarization using combination of deep neural networks equipped with auxiliary attention mechanism

Assessment of Transformer-Based Encoder-Decoder Model for Human-Like Summarization

Sharing Pre-Trained Bert Decoder For A Hybrid Summarization