STRUM-LLM: Attributed and Structured Contrastive Summarization

Beliz Gunel,James B. Wendt,Jing Xie,Yichao Zhou,Nguyen Vo,Zachary Fisher,Sandeep Tata

2024-03-26

Abstract:Users often struggle with decision-making between two options (A vs B), as it usually requires time-consuming research across multiple web pages. We propose STRUM-LLM that addresses this challenge by generating attributed, structured, and helpful contrastive summaries that highlight key differences between the two options. STRUM-LLM identifies helpful contrast: the specific attributes along which the two options differ significantly and which are most likely to influence the user's decision. Our technique is domain-agnostic, and does not require any human-labeled data or fixed attribute list as supervision. STRUM-LLM attributes all extractions back to the input sources along with textual evidence, and it does not have a limit on the length of input sources that it can process. STRUM-LLM Distilled has 100x more throughput than the models with comparable performance while being 10x smaller. In this paper, we provide extensive evaluations for our method and lay out future directions for our currently deployed system.

Computation and Language,Artificial Intelligence,Information Retrieval,Machine Learning

What problem does this paper attempt to address?

This paper attempts to solve the problem that when making a decision between two options (A vs B), users usually need to spend a great deal of time researching on multiple web pages. STRUM - LLM addresses this challenge by generating attributed, structured comparative summaries that can highlight the key differences between the two options and influence users' decisions. Specifically, STRUM - LLM can identify specific attributes that are helpful for decision - making, which have significant differences between the two options and are most likely to influence users' decisions. In addition, this method is not domain - specific, does not require any manually - annotated data or a fixed list of attributes as supervision, can handle input sources of arbitrary length, and all the extracted information can be traced back to the original source with textual evidence. The paper also presents the performance improvements of STRUM - LLM. For example, the throughput of STRUM - LLM Distilled is 100 times higher than that of a model with comparable performance, while its size is 10 times smaller.

STRUM-LLM: Attributed and Structured Contrastive Summarization

SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization.

ConVerSum: A Contrastive Learning based Approach for Data-Scarce Solution of Cross-Lingual Summarization Beyond Direct Equivalents

Align and Attend: Multimodal Summarization with Dual Contrastive Losses

DCDSum: An interpretable extractive summarization framework based on contrastive learning method

LaMSUM: Creating Extractive Summaries of User Generated Content using LLMs

UniSumEval: Towards Unified, Fine-Grained, Multi-Dimensional Summarization Evaluation for LLMs

TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale

Attribute Structuring Improves LLM-Based Evaluation of Clinical Text Summaries

On Learning to Summarize with Large Language Models as References

Learning to Summarize from LLM-generated Feedback

Sequence Level Contrastive Learning for Text Summarization

LLM Based Multi-Document Summarization Exploiting Main-Event Biased Monotone Submodular Content Extraction

Leveraging Salience Analysis and Sparse Attention for Long Document Summarization

Sentence salience contrastive learning for abstractive text summarization

A Comparative Study of Quality Evaluation Methods for Text Summarization

Comparative Analysis of Open-Source Language Models in Summarizing Medical Text Data

Dual-Level Contrastive Learning for Improving Conciseness of Summarization

Contrastive Attention Mechanism for Abstractive Sentence Summarization

Building Contrastive Summaries of Subjective Text Via Opinion Ranking

Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarization