Abstract:With the explosive growth of video data, video summarization, which attempts to seek the minimum subset of frames while still conveying the main story, has become one of the hottest topics. Nowadays, substantial achievements have been made by supervised learning techniques, especially after the emergence of deep learning. However, it is extremely expensive and difficult to collect human annotation for large-scale video datasets. To address this problem, we propose a convolutional attentive adversarial network (CAAN), whose key idea is to build a deep summarizer in an unsupervised way. Upon the generative adversarial network, our overall framework consists of a generator and a discriminator. The former predicts importance scores for all frames of a video while the latter tries to distinguish the score-weighted frame features from original frame features. Specifically, the generator employs a fully convolutional sequence network to extract global representation of a video, and an attention-based network to output normalized importance scores. To learn the parameters, our objective function is composed of three loss functions, which can guide the frame-level importance score prediction collaboratively. To validate this proposed method, we have conducted extensive experiments on two public benchmarks SumMe and TVSum. The results show the superiority of our proposed method against other state-of-the-art unsupervised approaches. Our method even outperforms some published supervised approaches.

Learning Multiscale Hierarchical Attention for Video Summarization

A Human-Machine Collaborative Video Summarization Framework Using Pupillary Response Signals

An Unsupervised Video Summarization Method Based on Multimodal Representation.

Hierarchical organization for medical video summarization using latent visual and semantic analysis

A Hierarchical Spatial–Temporal Cross-Attention Scheme for Video Summarization Using Contrastive Learning

Video Summarization with a Dual-Path Attentive Network

MHSCNet: A Multimodal Hierarchical Shot-aware Convolutional Network for Video Summarization

Supervised Video Summarization via Multiple Feature Sets with Parallel Attention

Convolutional Hierarchical Attention Network for Query-Focused Video Summarization.

Deep Semantic and Attentive Network for Unsupervised Video Summarization

Multi-Level Spatiotemporal Network for Video Summarization

Video Summarization with Attention-Based Encoder-Decoder Networks.

Spatial Attention Model‐modulated Bi‐directional Long Short‐term Memory for Unsupervised Video Summarisation

Unsupervised Video Summarization with a Convolutional Attentive Adversarial Network

Spatiotemporal Two-Stream LSTM Network for Unsupervised Video Summarization

HSA-RNN: Hierarchical Structure-Adaptive RNN for Video Summarization

Weakly Supervised Video Summarization by Hierarchical Reinforcement Learning

Video Summarization Via Weighted Neighborhood Based Representation.

Deep Attentive Video Summarization with Distribution Consistency Learning

Hierarchical multi‐modal video summarization with dynamic sampling

Graph Attention Networks Adjusted Bi-LSTM for Video Summarization