Abstract:The proliferation of video content on platforms like YouTube and Vimeo presents significant challenges in efficiently locating relevant information. Automatic video summarization aims to address this by extracting and presenting key content in a condensed form. This thesis explores enhancing video summarization by integrating text-based queries and conditional modeling to tailor summaries to user needs. Traditional methods often produce fixed summaries that may not align with individual requirements. To overcome this, we propose a multi-modal deep learning approach that incorporates both textual queries and visual information, fusing them at different levels of the model architecture. Evaluation metrics such as accuracy and F1-score assess the quality of the generated summaries. The thesis also investigates improving text-based query representations using contextualized word embeddings and specialized attention networks. This enhances the semantic understanding of queries, leading to better video summaries. To emulate human-like summarization, which accounts for both visual coherence and abstract factors like storyline consistency, we introduce a conditional modeling approach. This method uses multiple random variables and joint distributions to capture key summarization components, resulting in more human-like and explainable summaries. Addressing data scarcity in fully supervised learning, the thesis proposes a segment-level pseudo-labeling approach. This self-supervised method generates additional data, improving model performance even with limited human-labeled datasets. In summary, this research aims to enhance automatic video summarization by incorporating text-based queries, improving query representations, introducing conditional modeling, and addressing data scarcity, thereby creating more effective and personalized video summaries.

Capturing User Reading Behaviors for Personalized Document Summarization

A Human-Machine Collaborative Video Summarization Framework Using Pupillary Response Signals

Creating Personalized Video Summaries Via Semantic Event Detection

An Interactive Personalized Video Summarization Based on Sketches.

Personalized Text Summarization Based on Gaze Patterns

SumRecom: A Personalized Summarization Approach by Learning from Users' Feedback

Collaborative Summarization: when Collaborative Filtering Meets Document Summarization.

Personalized Summarization of Scientific Scholarly Texts

I Want This, Not That: Personalized Summarization of Scientific Scholarly Texts

Personalized Video Summarization by Multimodal Video Understanding

PersonalSum: A User-Subjective Guided Personalized Summarization Dataset for Large Language Models

Generating extractive sentiment summaries for natural language user queries on products

Towards Personalized and Human-in-the-Loop Document Summarization

Eusum: Extracting Easy-To-Understand English Summaries For Non-Native Readers

UserSumBench: A Benchmark Framework for Evaluating User Summarization Approaches

Auto-summarization of multimedia meeting records based on accessing log

Automatic Multi-Document Summarization for Digital Libraries

SummAct: Uncovering User Intentions Through Interactive Behaviour Summarisation

Document Summarization Via Self-Present Sentence Relevance Model.

Personalized Video Summarization using Text-Based Queries and Conditional Modeling

Readability Controllable Biomedical Document Summarization