Query-oriented unsupervised multi-document summarization via deep learning model

ZhongSheng-hua,LiuYan,LiBin,LongJing

IF: 8.5

2015-01-01

Expert Systems with Applications

Abstract:First attempt of deep learning for query-oriented multi-document summarization.Novel algorithm pushes out important concepts layer by layer effectively.Confirm excellent extraction ability under un...

What problem does this paper attempt to address?

Query-oriented unsupervised multi-document summarization via deep learning model

Sheng-hua Zhong,Yang Liu,Bin Li,Jing Long

DOI: https://doi.org/10.1016/j.eswa.2015.05.034

IF: 8.5

2015-01-01

Expert Systems with Applications

Abstract:Capturing the compositional process from words to documents is a key challenge in natural language processing and information retrieval: Extractive style query-oriented multi-document summarization generates a summary by extracting a proper set of sentences from multiple documents based on pre-given query. This paper proposes a novel document summarization framework based on deep learning model, which has been shown outstanding extraction ability in many real-world applications. The framework consists of three parts: concepts extraction, summary generation, and reconstruction validation. A new query-oriented extraction technique is proposed to extract information distributed in multiple documents. Then, the whole deep architecture is fine-tuned by minimizing the information loss in reconstruction validation. According to the concepts extracted from deep architecture layer by layer, dynamic programming is used to seek most informative set of sentences for the summary. Experiment on three benchmark datasets (DUC 2005, 2006, and 2007) assess and confirm the effectiveness of the proposed framework and algorithms. Experiment results show that the proposed method outperforms state-of-the-art extractive summarization approaches. Moreover, we also provide the statistical analysis of query words based on Amazon's Mechanical Turk (MTurk) crowdsourcing platform. There exists underlying relationships from topic words to the content which can contribute to summarization task. (C) 2015 Elsevier Ltd. All rights reserved.
Query-focused Multi-document Summarization: Combining a Novel Topic Model with Graph-based Semi-supervised Learning

Jiwei Li,Sujian Li

DOI: https://doi.org/10.48550/arxiv.1212.2036

2012-01-01

Abstract:Graph-based semi-supervised learning has proven to be an effective approach for query-focused multi-document summarization. The problem of previous semi-supervised learning is that sentences are ranked without considering the higher level information beyond sentence level. Researches on general summarization illustrated that the addition of topic level can effectively improve the summary quality. Inspired by previous researches, we propose a two-layer (i.e. sentence layer and topic layer) graph-based semi-supervised learning approach. At the same time, we propose a novel topic model which makes full use of the dependence between sentences and words. Experimental results on DUC and TAC data sets demonstrate the effectiveness of our proposed approach.
Automatic Document Summarization Via Deep Neural Networks

Chengwei Yao,Jianfen Shen,Gencai Chen

DOI: https://doi.org/10.1109/iscid.2015.83

2015-01-01

Abstract:Automatic document summarization aim to extracting sentences which might cover the main content of a document or documents. To achieve this, many algorithms have been tried to rank the sentences by using task-specific features in a shallow architecture. The main challenge of these approaches is to keep balance between information coverage and redundancy because of absence of discovering the intrinsic semantic representation. Inspired by the recent successful achievement of Deep Learning, this paper proposes a new framework of document summarization via Deep Neural Networks (DNNs). Specifically, we feed the sentences as the input to the visible layer of DNNs. After pretraining layer by layer and fine-tuning, the lower dimensional semantic space can be revealed. Based on this space, we design sentences extraction algorithm to construct the summary. Experiments on the DUC2006 and DUC2007 dataset show that our framework works better than state-of-the-art methods.
An Unsupervised Video Summarization Method Based on Multimodal Representation.

Zhuo Lei,Qiang Yu,Lidan Shou,Shengquan Li,Yunqing Mao

DOI: https://doi.org/10.1007/978-981-99-4761-4_15

2023-01-01

Abstract:A good video summary should convey the whole story and feature the most important content. However, the importance of video content is often subjective, and users should have the option to personalize the summary by using natural language to specify what is important to them. Moreover, existing methods usually apply only visual cues to solve generic video summarization tasks, while this work introduces a single unsupervised multi-modal framework for addressing both generic and query-focused video summarization. We use a multi-head attention model to represent the multi-modal feature. We apply a Transformer-based model to learn the frame scores based on their representative, diversity and reconstruction losses. Especially, we develop a novel representative loss to train the model based on both visual and semantic information. We outperform previous state-of-the-art work with superior results on both generic and query-focused video summarization datasets.
Exploring hypergraph-based semi-supervised ranking for query-oriented summarization

Wei Wang,Sujian Li,Jiwei Li,Wenjie Li,Furu Wei

DOI: https://doi.org/10.1016/j.ins.2013.03.012

IF: 8.1

2013-01-01

Information Sciences

Abstract:Traditional graph based sentence ranking algorithms such as LexRank and HITS model the documents to be summarized as a text graph where nodes represent sentences and edges represent pairwise relations. Such modeling cannot capture complex group relationship shared among multiple sentences which can be useful for sentence ranking. In this paper, we propose to take advantage of hypergraph to remedy this defect. In a text hypergraph, nodes still represent sentences, yet hyperedges are allowed to connect more than two sentences. With a text hypergraph, we are thus able to integrate both group relationship and pairwise relationship into a unified framework. Then, a hypergraph based semi-supervised sentence ranking algorithm is developed for query-oriented extractive summarization, where the influence of query is propagated to sentences through the structure of the constructed text hypergraph. When evaluated on DUC datasets, performance of our proposed approach shows improvements compared to a number of baseline systems.
A Novel Feature-based Bayesian Model for Query Focused Multi-document Summarization

Jiwei Li,Sujian Li

DOI: https://doi.org/10.1162/tacl_a_00212

IF: 10.9

2013-01-01

Transactions of the Association for Computational Linguistics

Abstract:Both supervised learning methods and LDA based topic model have been successfully applied in the field of query focused multi-document summarization. In this paper, we propose a novel supervised approach that can incorporate rich sentence features into Bayesian topic models in a principled way, thus taking advantages of both topic model and feature based supervised learning methods. Experiments on TAC2008 and TAC2009 demonstrate the effectiveness of our approach.
An Unsupervised Multi-Document Summarization Framework Based on Neural Document Model.

Shulei Ma,Zhi-Hong Deng,Yunlun Yang

2016-01-01

Abstract:In the age of information exploding, multi-document summarization is attracting particular attention for the ability to help people get the main ideas in a short time. Traditional extractive methods simply treat the document set as a group of sentences while ignoring the global semantics of the documents. Meanwhile, neural document model is effective on representing the semantic content of documents in low-dimensional vectors. In this paper, we propose a document-level reconstruction framework named DocRebuild, which reconstructs the documents with summary sentences through a neural document model and selects summary sentences to minimize the reconstruction error. We also apply two strategies, sentence filtering and beamsearch, to improve the performance of our method. Experimental results on the benchmark datasets DUC 2006 and DUC 2007 show that DocRebuild is effective and outperforms the state-of-the-art unsupervised algorithms.
Multi-document Summarization via Deep Learning Techniques: A Survey

Congbo Ma,Wei Emma Zhang,Mingyu Guo,Hu Wang,Quan Z. Sheng

DOI: https://doi.org/10.48550/arXiv.2011.04843

2021-12-09

Abstract:Multi-document summarization (MDS) is an effective tool for information aggregation that generates an informative and concise summary from a cluster of topic-related documents. Our survey, the first of its kind, systematically overviews the recent deep learning based MDS models. We propose a novel taxonomy to summarize the design strategies of neural networks and conduct a comprehensive summary of the state-of-the-art. We highlight the differences between various objective functions that are rarely discussed in the existing literature. Finally, we propose several future directions pertaining to this new and exciting field.

Computation and Language,Machine Learning
Query-focused Multi-Document Summarization: Combining a Topic Model with Graph-based Semi-supervised Learning.

Yanran Li,Sujian Li

2014-01-01

Abstract:Graph-based learning algorithms have been shown to be an effective approach for query-focused multi-document summarization (MDS). In this paper, we extend the standard graph ranking algorithm by proposing a two-layer (i.e. sentence layer and topic layer) graph-based semi-supervised learning approach based on topic modeling techniques. Experimental results on TAC datasets show that by considering topic information, we can effectively improve the summary performance.
An Unsupervised Extractive Summarization Method Based on Multi-Round Computation

Dehao Tao,Ya Xiong,Jin He,Skevin,Yongfeng Huang

2021-01-01

Abstract:Text summarization methods have attracted much attention all the time. In recent years, deep learning has been applied to text summarization, and it turned out to be pretty effective. However, most of the current text summarization methods based on deep learning need large-scale datasets, which is difficult to achieve in practical applications. In this paper, an unsupervised extractive text summarization method based on multi-round calculation is proposed. Based on the directed graph algorithm, we change the traditional method of calculating the sentence ranking at one time to multi-round calculation, and the summary sentences are dynamically optimized after each round of calculation to better match the characteristics of the text. In this paper, experiments are carried out on four data sets, each separately containing Chinese, English, long and short texts. The experiment results show that our method has better performance than both baseline methods and other unsupervised methods and is robust on different datasets.
Query-Focused Summarization by Combining Topic Model and Affinity Propagation

Dewei Chen,Jie Tang,Limin Yao,Lizhu Zhou

DOI: https://doi.org/10.1007/978-3-642-00672-2_17

2009-01-01

Abstract:The goal of query-focused summarization is to extract a summary for a given query from the document collection. Although much work has been done for this problem, there are still many challenging issues: (1) The length of the summary is predefined by, for example, the number of word tokens or the number of sentences. (2) A query usually asks for information of several perspectives (topics); however existing methods cannot capture topical aspects with respect to the query. In this paper, we propose a novel approach by combining statistical topic model and affinity propagation. Specifically, the topic model, called qLDA, can simultaneously model documents and the query. Moreover, the affinity propagation can automatically discover key sentences from the document collection without predefining the length of the summary. Experimental results on DUC05 and DUC06 data sets show that our approach is effective and the summarization performance is better than baseline methods.
Multi-Document Abstractive Summarization Using Chunk-graph and Recurrent Neural Network

Jianwei Niu,Huan Chen,Qingjuan Zhao,Limin Sun,Mohammed Atiquzzaman

DOI: https://doi.org/10.1109/icc.2017.7996331

2017-01-01

Abstract:Automatic multi-document abstractive summarization system is used to summarize several documents into a short one with generated new sentences. Many of them are based on word-graph and ILP method, and lots of sentences are ignored because of the heavy computation load. To reduce computation and generate readable and informative summaries, we propose a novel abstractive multi-document summarization system based on chunk-graph (CG) and recurrent neural network language model (RNNLM). In our approach, A CG which is based on word-graph is constructed to organize all information in a sentence cluster, CG can reduce the size of graph and keep more semantic information than word-graph. We use beam search and character-level RNNLM to generate readable and informative summaries from the CG for each sentence cluster, RNNLM is a better model to evaluate sentence linguistic quality than n-gram language model. Experimental results show that our proposed system outperforms all baseline systems and reach the state-of-art systems, and the system with CG can generate better summaries than that with ordinary word-graph.
Query-focused Multi-Document Summarization Based on Query-Sensitive Feature Space

Wenpeng Yin,Yulong Pei,Fan Zhang,Lian'en Huang

DOI: https://doi.org/10.1145/2396761.2398491

2012-01-01

Abstract:Query-oriented relevance, information richness and novelty are important requirements in query-focused summarization, which, to a considerable extent, determine the summary quality. Previous work either rarely took into account all above demands simultaneously or dealt with part of them in the dynamic process of choosing sentences to generate a summary. In this paper, we propose a novel approach that integrates all these requirements skillfully by treating them as sentence features, making that the finally generated summary could fully reflect the combinational effect of these properties. Experimental results on the DUC2005 and DUC2006 datasets demonstrate the effectiveness of our approach.
Query-Oriented Micro-Video Summarization

Mengzhao Jia,Yinwei Wei,Xuemeng Song,Teng Sun,Min Zhang,Liqiang Nie

DOI: https://doi.org/10.1109/tpami.2024.3355402

IF: 23.6

2024-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:Query-oriented micro-video summarization task aims to generate a concise sentence with two properties: (a) summarizing the main semantic of the micro-video and (b) being expressed in the form of search queries to facilitate retrieval. Despite its enormous application value in the retrieval area, this direction has barely been explored. Previous studies of summarization mostly focus on the content summarization for traditional long videos. Directly applying these studies is prone to gain unsatisfactory results because of the unique features of micro-videos and queries: diverse entities and complex scenes within a short time, semantic gaps between modalities, and various queries in distinct expressions. To specifically adapt to these characteristics, we propose a query-oriented micro-video summarization model, dubbed QMS. It employs an encoder-decoder-based transformer architecture as the skeleton. The multi-modal (visual and textual) signals are passed through two modal-specific encoders to obtain their representations, followed by an entity-aware representation learning module to identify and highlight critical entity information. As to the optimization, regarding the large semantic gaps between modalities, we assign different confidence scores according to their semantic relevance in the optimization process. Additionally, we develop a novel strategy to sample the effective target query among the diverse query set with various expressions. Extensive experiments demonstrate the superiority of the QMS scheme, on both the summarization and retrieval tasks, over several state-of-the-art methods.

computer science, artificial intelligence,engineering, electrical & electronic
AttSum: Joint Learning of Focusing and Summarization with Neural Attention.

Ziqiang Cao,Wenjie Li,Sujian Li,Furu Wei,Yanran Li

2016-01-01

Abstract:Query relevance ranking and sentence saliency ranking are the two main tasks in extractive query-focused summarization. Previous supervised summarization systems often perform the two tasks in isolation. However, since reference summaries are the trade-off between relevance and saliency, using them as supervision, neither of the two rankers could be trained well. This paper proposes a novel summarization system called AttSum, which tackles the two tasks jointly. It automatically learns distributed representations for sentences as well as the document cluster. Meanwhile, it applies the attention mechanism to simulate the attentive reading of human behavior when a query is given. Extensive experiments are conducted on DUC query-focused summarization benchmark datasets. Without using any hand-crafted features, AttSum achieves competitive performance. It is also observed that the sentences recognized to focus on the query indeed meet the query need.
Domain Adaptation and Summary Distillation for Unsupervised Query Focused Summarization

Jiancheng Du,Yang Gao

DOI: https://doi.org/10.1109/TKDE.2023.3296441

IF: 9.235

2024-03-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:Text summarizing is the task of reducing a document's length while maintaining its essential information. In the age of information explosion, how to obtain the content that users needed from a large volume of information becomes particularly significant. Under such circumstances, query-focused abstractive summarization (qfs) becomes more dominant since it is able to focus on user needs while delivering fluent, concise, succinct paraphrased summaries. However, unlike generic summarization, which has achieved remarkable progress driven by a substantial amount of parallel data, the qfs struggles due to a deficiency of parallel corpus. Therefore, in this paper, we leverage a typical large generic summarization dataset to facilitate the pressing demands on unsupervised qfs. The large-scale query-free benchmark is automatically transformed into a query-focused dataset (Query-CNNDM) while preserving its informative summaries. We propose a simple yet effective unsupervised method, called Domain Adaptation and Summary Distillation method (DASD). In the model, to achieve the domain adaptation for unsupervised qfs, we design a query-aware gap sentence generation (q-GSG) strategy to equip the model with the capability of learning target textual knowledge and obtaining a good initialization at the target domain. As instance-specific regularization, we train a teacher model with the Query-CNNDM to generate pseudo-labels for summary distillation. Experimental results indicate that our DASD model achieves state-of-the-art performance on two benchmark datasets, Debatepedia and Wikiref, in a zero-shot setting and shows good generalization to the abstractive few-shot qfs.

Computer Science
Exploring Neural Models for Query-Focused Summarization

Jesse Vig,Alexander R. Fabbri,Wojciech Kryściński,Chien-Sheng Wu,Wenhao Liu

DOI: https://doi.org/10.48550/arXiv.2112.07637

2021-12-14

Computation and Language

Abstract:Query-focused summarization (QFS) aims to produce summaries that answer particular questions of interest, enabling greater user control and personalization. While recently released datasets, such as QMSum or AQuaMuSe, facilitate research efforts in QFS, the field lacks a comprehensive study of the broad space of applicable modeling methods. In this paper we conduct a systematic exploration of neural approaches to QFS, considering two general classes of methods: two-stage extractive-abstractive solutions and end-to-end models. Within those categories, we investigate existing models and explore strategies for transfer learning. We also present two modeling extensions that achieve state-of-the-art performance on the QMSum dataset, up to a margin of 3.38 ROUGE-1, 3.72 ROUGE2, and 3.28 ROUGE-L when combined with transfer learning strategies. Results from human evaluation suggest that the best models produce more comprehensive and factually consistent summaries compared to a baseline model. Code and checkpoints are made publicly available: https://github.com/salesforce/query-focused-sum.
Topic aspect analysis for multi-document summarization.

Chao Shen,Dingding Wang,Tao Li

DOI: https://doi.org/10.1145/1871437.1871668

2010-01-01

Abstract:Query-based multi-document summarization aims to create a short summary given a collection of documents and a query. Most of the existing methods treat the query as one single sentence and rank the sentences in the documents based on their similarities with the query sentence. However, these methods lack of intensive analysis on the given query which typically consist of several topic aspects. In this paper, we propose a topic aspect extraction method to discover the aspect words and sentences contained in the query narrative texts and the input documents, and then incorporate these aspect words and sentences into a cross propagation model based on the sentence-term bipartite graph for document summarization. Experiments on DUC benchmark data show the effectiveness of our proposed approach on the topic-driven document summarization task.
Query-Focused Multi-document Summarization Based on Concept Importance

Hai-Tao Zheng,Ji-Min Guo,Yong Jiang,Shu-Tao Xia

DOI: https://doi.org/10.1007/978-3-319-31750-2_35

2016-01-01

Abstract:With the exponential growth of the web documents and the requirement of limited bandwidth for mobile devices, it becomes more and more difficult for users to get information they look forward to from the vast amount of information. Query-focused summarization gets more attention from both the research and engineering area in recent years. However, existing query-focused summarization methods don’t consider the conceptual relation and the concept importance that make up the sentences, a concept is the title of a wikipedia article and can express an entity or action. In this article. We propose a novel method called Query-focused Multi-document Summarization based on Concept Importance (QMSCI). We first map sentence to concepts and get ranked weighted concepts by reinforcement between the concepts of sentences and concepts of the query in a bipartite graph, then we use the ranked weighted concepts to help to rank the sentences in a hyper-graph model, sentences that contain important concepts, related with the query and also central among sentences are ranked higher and comprise the summary. We experiment on the DUC datasets, the experimental result demonstrates the effectiveness of our proposed method compared to the state-of-art methods.
Deep Dependency Substructure-Based Learning for Multidocument Summarization.

Su Yan,Xiaojun Wan

DOI: https://doi.org/10.1145/2766447

2015-01-01

Abstract:Most extractive style topic-focused multidocument summarization systems generate a summary by ranking textual units in multiple documents and extracting a proper subset of sentences biased to the given topic. Usually, the textual units are simply represented as sentences or n-grams, which do not carry deep syntactic and semantic information. This article presents a novel extractive topic-focused multidocument summarization framework. The framework proposes a new kind of more meaningful and informative units named frequent Deep Dependency Sub-Structure (DDSS) and a topic-sensitive Multi-Task Learning (MTL) model for frequent DDSS ranking. Given a document set, first, we parse all the sentences into deep dependency structures with a Head-driven Phrase Structure Grammar (HPSG) parser and mine the frequent DDSSs after semantic normalization. Then we employ a topic-sensitive MTL model to learn the importance of these frequent DDSSs. Finally, we exploit an Integer Linear Programming (ILP) formulation and use the frequent DDSSs as the essentials for summary extraction. Experimental results on two DUC datasets demonstrate that our proposed approach can achieve state-of-the-art performance. Both the DDSS information and the topic-sensitive MTL model are validated to be very helpful for topic-focused multidocument summarization.

Query-oriented unsupervised multi-document summarization via deep learning model

Query-oriented unsupervised multi-document summarization via deep learning model

Query-focused Multi-document Summarization: Combining a Novel Topic Model with Graph-based Semi-supervised Learning

Automatic Document Summarization Via Deep Neural Networks

An Unsupervised Video Summarization Method Based on Multimodal Representation.

Exploring hypergraph-based semi-supervised ranking for query-oriented summarization

A Novel Feature-based Bayesian Model for Query Focused Multi-document Summarization

An Unsupervised Multi-Document Summarization Framework Based on Neural Document Model.

Multi-document Summarization via Deep Learning Techniques: A Survey

Query-focused Multi-Document Summarization: Combining a Topic Model with Graph-based Semi-supervised Learning.

An Unsupervised Extractive Summarization Method Based on Multi-Round Computation

Query-Focused Summarization by Combining Topic Model and Affinity Propagation

Multi-Document Abstractive Summarization Using Chunk-graph and Recurrent Neural Network

Query-focused Multi-Document Summarization Based on Query-Sensitive Feature Space

Query-Oriented Micro-Video Summarization

AttSum: Joint Learning of Focusing and Summarization with Neural Attention.

Domain Adaptation and Summary Distillation for Unsupervised Query Focused Summarization

Exploring Neural Models for Query-Focused Summarization

Topic aspect analysis for multi-document summarization.

Query-Focused Multi-document Summarization Based on Concept Importance

Deep Dependency Substructure-Based Learning for Multidocument Summarization.