Abstract:Automatic summarization plays an important role in the exponential document growth on the Web. On content websites such as <a class="link-external link-http" href="http://CNN.com" rel="external noopener nofollow">this http URL</a> and <a class="link-external link-http" href="http://WikiHow.com" rel="external noopener nofollow">this http URL</a>, there often exist various kinds of side information along with the main document for attention attraction and easier understanding, such as videos, images, and queries. Such information can be used for better summarization, as they often explicitly or implicitly mention the essence of the article. However, most of the existing side-aware summarization methods are designed to incorporate either single-modal or multi-modal side information, and cannot effectively adapt to each other. In this paper, we propose a general summarization framework, which can flexibly incorporate various modalities of side information. The main challenges in designing a flexible summarization model with side information include: (1) the side information can be in textual or visual format, and the model needs to align and unify it with the document into the same semantic space, (2) the side inputs can contain information from various aspects, and the model should recognize the aspects useful for summarization. To address these two challenges, we first propose a unified topic encoder, which jointly discovers latent topics from the document and various kinds of side information. The learned topics flexibly bridge and guide the information flow between multiple inputs in a graph encoder through a topic-aware interaction. We secondly propose a triplet contrastive learning mechanism to align the single-modal or multi-modal information into a unified semantic space, where the summary quality is enhanced by better understanding the document and side information. Results show that our model significantly surpasses strong baselines on three public single-modal or multi-modal benchmark summarization datasets.

RTSUM: Relation Triple-based Interpretable Summarization with Multi-level Salience Visualization

An Unsupervised Video Summarization Method Based on Multimodal Representation.

UniSumEval: Towards Unified, Fine-Grained, Multi-Dimensional Summarization Evaluation for LLMs

Text Summarization Based on Sentence Selection with Semantic Representation

SgSum: Transforming Multi-document Summarization into Sub-graph Selection

VSumVis: Interactive Visual Understanding and Diagnosis of Video Summarization Model

Multisumm: Towards A Unified Model For Multi-Lingual Abstractive Summarization

Multi-Dimensional Optimization for Text Summarization via Reinforcement Learning

Question-focused Summarization Based on Semantic Relational Triple

TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale

Information-Theoretic Distillation for Reference-less Summarization

OHiFormer: Object-Wise Hierarchical Dependency-Based Transformer for Screen Summarization

A Novel Relational Learning-To-Rank Approach For Topic-Focused Multi-Document Summarization

SummVis: Interactive Visual Analysis of Models, Data, and Evaluation for Text Summarization

A Topic-aware Summarization Framework with Different Modal Side Information

A Supervised Aggregation Framework for Multi-Document Summarization.

Unsupervised Document Summarization from Data Reconstruction Perspective.

RankSum An unsupervised extractive text summarization based on rank fusion

Multi-Document Summarization Based On Two-Level Sparse Representation Model

Multi-Document Summarization via Discriminative Summary Reranking

RelationListwise for Query-Focused Multi-Document Summarization.