Abstract:Video summarization (VS) is generally formulated as a subset selection problem where a set of representative keyframes or key segments is selected from an entire video frame set. Though many sparse subset selection based VS algorithms have been proposed in the past decade, most of them adopt linear sparse formulation in the explicit feature vector space of video frames, and don't consider the local or global relationships among frames. In this paper, we first extend the conventional sparse subset selection for VS into kernel block sparse subset selection (KBS3) to utilize the advantage of kernel sparse coding and introduce a local inter-frame relationship through packing of frame blocks. Going a step further, we propose a similarity based block sparse subset selection (SB2S3) model by applying a specially designed transformation matrix on the KBS3 model in order to introduce a kind of global inter-frame relationship through the similarity. Finally, a greedy pursuit based algorithm is devised for the proposed NP-hard model optimization. The proposed SB2S3 has the following advantages: 1) through the similarity between each frame and any other frame, the global relationship among all frames can be considered; 2) through block sparse coding, the local relationship of adjacent frames is further considered; and 3) it has a wider application, since features can derive similarity, but not vice versa. It is believed that the effect of modeling such global and local relationships among frames in this paper, is similar to that of modeling the long-range and short-range dependencies among frames in deep learning based methods. Experimental results on three benchmark datasets have demonstrated that the proposed approach is superior to not only other sparse subset selection based VS methods but also most unsupervised deep-learning based VS methods.

Summarizing Audiovisual Contents of A Video Program

Creating Motion Video Summaries with Partial Audio-Visual Alignment

A Novel Compact Yet Rich Key Frame Creation Method for Compressed Video Summarization

Text Semantics Based Automatic Summarization for Chinese Videos

Hierarchical organization for medical video summarization using latent visual and semantic analysis

AudioVisual Video Summarization

Attention-Based Audio-Visual Fusion For Video Summarization

Video Summarization Via Simultaneous Block Sparse Representation.

VIDEO SUMMARIZATION VIA TEMPORAL COLLABORATIVE REPRESENTATION OF ADJACENT FRAMES

Multimodal-Based and Aesthetic-Guided Narrative Video Summarization

Similarity Based Block Sparse Subset Selection for Video Summarization.

An Aesthetic-Guided Multimodal Framework for Video Summarization

VideoXum: Cross-modal Visual and Textural Summarization of Videos

Automatic Video Summarization by Affinity Propagation Clustering and Semantic Content Mining

Video Presentation Board : A Semantic Visualization of Video Sequence

Hierarchical video content description and summarization using unified semantic and visual similarity

VSumVis: Interactive Visual Understanding and Diagnosis of Video Summarization Model

Personalized Video Summarization by Multimodal Video Understanding

Visual Summarization of Scholarly Videos using Word Embeddings and Keyphrase Extraction

A Knowledge Augmented and Multimodal-Based Framework for Video Summarization

Video Summarization Overview