Abstract:Video summarization is an important technique to browse, manage and retrieve a large amount of videos efficiently. The main objective of video summarization is to minimize the information loss when selecting a subset of video frames from the original video, hence the summary video can faithfully represent the overall story of the original video. Recently developed unsupervised video summarization approaches are free of requiring tedious annotation on important frames to train a video summarization model and thus are practically attractive. However, their performance is still limited due to the difficulty of minimizing information loss between the summary and original videos. In this paper, we address unsupervised video summarization by developing a novel Cycle-consistent Adversarial LSTM architecture to effectively reduce the information loss in the summary video. The proposed model, named Cycle-SUM, consists of a frame selector and a cycle-consistent learning based evaluator. The selector is a bi-directional LSTM network to capture the long-range relationship between video frames. To overcome the difficulty of specifying a suitable information preserving metric between original video and summary video, the evaluator is introduced to "supervise" selector to improve the video summarization quality. Specifically, the evaluator is composed of two generative adversarial networks (GANs), in which the forward GAN component is learned to reconstruct the original video from summary video, while the backward GAN learns to invert the process. We establish the relation between mutual information maximization and such cycle learning procedure and further introduce cycle-consistent loss to regularize the summarization. Extensive experiments on three video summarization benchmark datasets demonstrate a state-of-the-art performance, and show the superiority of the Cycle-SUM model compared with other unsupervised approaches.

Video Summarization Generation Model Based on Transformer and Deep Reinforcement Learning

A GAN Based Video Summarization Method with Representation Loss

An Unsupervised Video Summarization Method Based on Multimodal Representation.

Action Parsing-Driven Video Summarization Based on Reinforcement Learning

Video Summarization with Long Short-term Memory

Category Driven Deep Recurrent Neural Network for Video Summarization

Unsupervised Video Summarization via Reinforcement Learning and a Trained Evaluator

From Thumbnails to Summaries - A single Deep Neural Network to Rule Them All

Deep Learning Models for Automatic Summarization

Video Summarisation by Classification with Deep Reinforcement Learning

Attention is all you need for Videos: Self-attention based Video Summarization using Universal Transformers

Progressive Reinforcement Learning for Video Summarization

Deep Semantic and Attentive Network for Unsupervised Video Summarization

Video Summarization Generation Network Based on Dynamic Graph Contrastive Learning and Feature Fusion

Unsupervised Video Summarization With Cycle-Consistent Adversarial LSTM Networks

Improving Automatic Source Code Summarization Via Deep Reinforcement Learning

Hierarchical multimodal transformer to summarize videos

Video Summarization through Reinforcement Learning with a 3D Spatio-Temporal U-Net

Unsupervised Video Summarization with a Convolutional Attentive Adversarial Network

Video Joint Modelling Based on Hierarchical Transformer for Co-summarization

Cycle-SUM: Cycle-Consistent Adversarial LSTM Networks for Unsupervised Video Summarization.