Applications of Geometry Processing

ChenTao,LuAidong,HuShi-Min

IF: 1.821

2012-01-01

Computers & Graphics

Abstract:This paper presents a video summarization approach that automatically extracts and visualizes movie storylines in a static image for the purposes of efficient representation and quick overview. A n...

What problem does this paper attempt to address?

Description and Browsing of Video Story Structure

WANG Dong-hui,QIAN Hui,ZHU Miao-liang

DOI: https://doi.org/10.3321/j.issn:1000-436x.2005.03.021

2005-01-01

Abstract:This paper presented a method for the description and browsing of video story structure. An innovatory algorithm of video segmentation was introduced to find the shot edge. And another refined match algorithm based on content comparability was proposed to implement the map of video structure from one dimension to two dimensions. By describing video story structure as a storyboard and sequence structure graph, video content can be browsed summarily.
Visual Storylines: Semantic Visualization of Movie Sequence.

Tao Chen,Aidong Lu,Shi-Min Hu

DOI: https://doi.org/10.1016/j.cag.2012.02.010

IF: 1.821

2012-01-01

Computers & Graphics

Abstract:This paper presents a video summarization approach that automatically extracts and visualizes movie storylines in a static image for the purposes of efficient representation and quick overview. A new type of video visualization, Visual Storylines, is designed to summarize video storylines in an image composition while preserving the style of the original videos. This is achieved with a series of video analysis, image synthesis, relationship quantification and geometric layout optimization techniques. Specifically, we analyze the video contents and quantify the video story unit relationships automatically through clustering video shots according to both the visual and audio data. A multi-level storyline visualization method then organizes and synthesizes a suitable amount of representative information, including both the locations and interested objects and characters, with the assistance of arrows, according to the relationships between the video story units and the temporal structure of the video sequence. Several results have demonstrated that our approach is able to abstract the main storylines of professionally edited video such as commercial movies and TV series, though some semantic key clues might be missed in the summarization. Preliminary user studies have been performed to evaluate our approach, and the results show that our approach can be used to assist viewers to understand video contents when they are familiar with the context of the video or when a text synopsis is provided.
An Interactive Personalized Video Summarization Based on Sketches.

Yanqiu Zhang,Cui-Xia Ma,Jinkai Zhang,Dongliang Zhang,Yong-Jin Liu

DOI: https://doi.org/10.1145/2534329.2534343

2013-01-01

Abstract:Video summarization aims to give a condensed presentation of video content, facilitating users to efficiently and effectively browse videos and obtain specific information in a short time following users' attention. In this paper, we propose a novel method to generate personalized sketch summarization supporting sketch interaction that is characterized for being coherent, compact and interactive, allowing to measure how event tracks following the spatio-temporal progress and how users directly interact with the content of video. The stylized algorithm is provided to obtain sketches from key-frames and a graph is constructed representing attributes of sketches. Thus a layout algorithm based on graph is presented for generating sketch summarization. Furthermore, efficient interaction on sketch video summarization is provided. Finally, user experience shows that the proposed sketch summarization helps users quickly grasp the specific information, understand the context and interact with the content of interest.
Application of Geometry in Plural Space in physics

CAI Bi-wei,WU Chun-ming

DOI: https://doi.org/10.3969/j.issn.1008-8148.2013.04.023

2013-01-01

Abstract:A series of new concepts are introduced such as plural distance,visual length,physical length,plural angle,visual angle,physical angle,dual-diagram and so on. Application of Geometry in Plural Space in physics are systematically described.
Compounded Geometry Image Representation of Point Cloud Surfaces and Its Application

张明礼,张三元,陈志杨,叶修梓

DOI: https://doi.org/10.3785/j.issn.1008-973x.2004.12.005

2004-01-01

Abstract:To create geometry images for a point cloud surface, a novel approach for representing point cloud surfaces was presented. The Cartesian coordinates of 3D points were converted to spherical polar coordinates. By taking the two angular values as the 2D pixel position coordinates and the minimum positive radius value in the pixel as the gray value, a r(φ, θ) image was created. The left point cloud surface was converted into a θ(r, φ) image.In the same manner, a φ(r, θ) image was created. The three geometry images were then combined into a colored geometry image by mapping their gray values into R, G and B components respectively. Results show that the colored compound image can represent the original point cloud surface efficiently and accurately, so high ratio compression and simplification of the original point cloud surface can be achieved with image processing.
Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Hanqing Jiang,Guofeng Zhang,Huiyan Wang,Hujun Bao

DOI: https://doi.org/10.1109/tmm.2014.2368273

IF: 7.3

2015-01-01

IEEE Transactions on Multimedia

Abstract:Extracting spatio-temporally consistent segments from a video sequence is a challenging problem due to the complexity of color, motion and occlusions. Most existing spatio-temporal segmentation approaches have inherent difficulties in handling large displacement with significant occlusions. This paper presents a novel framework for spatio-temporal segmentation. With the estimated depth data beforehand by a multi-view stereo technique, we project the pixels to other frames for collecting the boundary and segmentation statistics in a video, and incorporate them into the segmentation energy for spatio-temporal optimization. In order to effectively solve this problem, we introduce an iterative optimization scheme by first initializing segmentation maps for each frame independently, and then link the correspondences among different frames and iteratively refine them with the collected statistics, so that a set of spatio-temporally consistent volume segments are finally achieved. The effectiveness and usefulness of our automatic framework are demonstrated via its applications for 3D reconstruction, video editing and semantic segmentation on a variety of challenging video examples.
Video Presentation Board : A Semantic Visualization of Video Sequence

Tao Chen,Ai-Dong Lu,Shi-Min Hu

2010-01-01

Abstract:This paper presents a video summarization method to visuali zing video sequences in a static image for the pur- poses of efficient representation and quick overview. It canassist viewers to understand important video contents by revealing essential information of video story units andtheir relations. We have designed a new method called Video Presentation Board to extract, organize, and synthes ize important video information in a succinct and mean- ingful visual format, which also preserves the elegance of o riginal videos. Specifically, we propose a new video shot clustering method using both visual and audio data for a nalyzing video contents and collecting important shot information. To represent these important video infor mation properly, we design a multi-level video sum- marization method to represent the contents and relations o f video shots through abstracting both locations and interested objects and characters. Suitable amount of sele cted video information, with the assistants of special visual languages, is then organized and synthesized accord ing to the relations between video events and temporal structure of the video sequence. We have designed and perfor med a preliminary user study to evaluate our ap- proach and collected very encouraging results. We believe t hat this approach seamlessly integrates video analysis and visualization methods to provide visually appealing re sults. Our approach can be used to assist viewers grasp video contents efficiently, and are especially suitable fordocumentations and reports.
Geometry Textures and Applications

Rodrigo de Toledo,Bin Wang,Bruno Levy

DOI: https://doi.org/10.1111/j.1467-8659.2008.01185.x

IF: 2.5

2008-01-01

Computer Graphics Forum

Abstract:Geometry textures are a novel geometric representation for surfaces based on height maps. The visualization is done through a graphics processing unit (GPU) ray casting algorithm applied to the whole object. At rendering time, the fine-scale details (mesostructures) are reconstructed preserving original quality. Visualizing surfaces with geometry textures allows a natural level-of-detail (LOD) behaviour. There are numerous applications that can benefit from the use of geometry textures. In this paper, besides a mesostructure visualization survey, we present geometry textures with three possible applications: rendering of solid models, geological surfaces visualization and surface smoothing.
An Effective Video Synopsis Approach with Seam Carving

Ke Li,Bo Yan,Weiyi Wang,Hamid Gharavi

DOI: https://doi.org/10.1109/LSP.2015.2496558

2016-01-01

IEEE Signal Processing Letters

Abstract:With a growth of surveillance cameras, the amount of captured videos expands. Manually analyzing and retrieving surveillance video is labor intensive and expensive. It would be much more convenient to generate a video digest, with which we can view the video in a fast and motion-preserving way. In this paper, we propose a novel video synopsis approach to generate condensed video, which uses an obj...
A Depth Extraction Method Based on Motion and Geometry for 2D to 3D Conversion

Xiaojun Huang,Lianghao Wang,Junjun Huang,Dongxiao Li,Ming Zhang

DOI: https://doi.org/10.1109/IITA.2009.481

2009-01-01

Abstract:With the development of 3DTV, the conversion of existing 2D videos to 3D videos becomes an important component of 3D content production. One of the key steps in 2D to 3D conversion is how to generate a dense depth map. In this paper, we propose a novel depth extraction method based on motion and geometric information for 2D to 3D conversion, which consists of two major depth extraction modules, the depth from motion and depth from geometrical perspective. The H.264 motion estimation result is utilized and cooperates with moving object detection to diminish block effect and generates a motion-based depth map. On the other hand, a geometry-based depth map is generated by edge detection and Hough transform. Finally, the motion-based depth map and the geometry-based depth map are integrated into one depth map by a depth fusion algorithm.
Automatically Generating Hierarchical Summary for Film Video

ZHAO Ya-qin,ZHOU Xian-zhong,HE Xin

DOI: https://doi.org/10.3969/j.issn.1006-8961.2007.08.017

2007-01-01

Abstract:It is important to properly organize the unstructured video data for content-based video analysis and retrieval. In this paper, we propose a unified approach for film video summarization based on the analysis of video structure and motion attention model. Video shots are firstly grouped into shot clusters. Afterwards, according to the characterization of film video scene, a temporally and spatially integrated strategy is presented to parse shot clusters into semantic scenes in terms of the definition of temporal relationships between two shot clusters. Finally, representative frames and highlight shots are selected from scenes by using motion attention model. The scheme offers an efficient mean for browsing and effectively retrieving film video.
Generating fluent tubes in video synopsis

Minlong Lu,Yueming Wang,Gang Pan

DOI: https://doi.org/10.1109/ICASSP.2013.6638063

2013-01-01

ICASSP

Abstract:Video synopsis is one of the effective techniques to build a short video representation preserving the essential activities for a long video. Existing methods usually have the problem that a continuous activity (tube) from a single moving object is separated to a few small pieces. In this paper, two schemes are proposed to generate fluent tubes for video synopsis. The Gaussian mixture model and a texture method are combined to detect more compact foreground with shadow removed. The foreground constitutes a set of initial trajectories. A particle filter tracker is used to concatenate two trajectories if they belong to the same foreground activity, which generates more fluent tubes for video synopsis. Experimental results on 4 videos show that our method produces better accuracies and visual effects in video synopsis.
Refilming with Depth-Inferred Videos

Guofeng Zhang,Zilong Dong,Jiaya Jia,Liang Wan,Tien-Tsin Wong,Hujun Bao

DOI: https://doi.org/10.1109/TVCG.2009.47

IF: 5.2

2009-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:Compared to still image editing, content-based video editing faces the additional challenges of maintaining the spatiotemporal consistency with respect to geometry. This brings up difficulties of seamlessly modifying video content, for instance, inserting or removing an object. In this paper, we present a new video editing system for creating spatiotemporally consistent and visually appealing refi...
Multi-video Summarization Using Complex Graph Clustering and Mining.

Jian Shao,Dongming Jiang,Mengru Wang,Hong Chen,Lu Yao

DOI: https://doi.org/10.2298/csis1001085s

2010-01-01

Computer Science and Information Systems

Abstract:Multi-video summarization is a great theoretical and technical challenge due to the wider diversity of topics in multi-video than single-video as well as the multi-modality nature of multi-video over multi-document. In this paper, we propose an approach to analyze both visual and textual features across a set of videos and to create a so-called circular storyboard composed of topic-representative keyframes and keywords. We formulate the generation of circular storyboard as a problem of complex graph clustering and mining, in which each separated shot from visual data and each extracted keyword from speech transcripts are first structured into a complex graph and grouped into clusters; hidden topics in the representative keyframes and keywords are then mined from clustered complex graph while at the same time maximizing the coverage of the summary over the original video set. We also design experiments to evaluate the effectiveness of our approach and the proposed approach shows a better performance than two other storyboard baselines.
Multi-View Video Summarization

Yanwei Fu,Yanwen Guo,Yanshu Zhu,Feng Liu,Chuanming Song,Zhi-Hua Zhou

DOI: https://doi.org/10.1109/tmm.2010.2052025

IF: 7.3

2010-01-01

IEEE Transactions on Multimedia

Abstract:Previous video summarization studies focused on monocular videos, and the results would not be good if they were applied to multi-view videos directly, due to problems such as the redundancy in multiple views. In this paper, we present a method for summarizing multi-view videos. We construct a spatio-temporal shot graph and formulate the summarization problem as a graph labeling task. The spatio-temporal shot graph is derived from a hypergraph, which encodes the correlations with different attributes among multi-view video shots in hyperedges. We then partition the shot graph and identify clusters of event-centered shots with similar contents via random walks. The summarization result is generated through solving a multi-objective optimization problem based on shot importance evaluated using a Gaussian entropy fusion scheme. Different summarization objectives, such as minimum summary length and maximum information coverage, can be accomplished in the framework. Moreover, multi-level summarization can be achieved easily by configuring the optimization parameters. We also propose the multi-view storyboard and event board for presenting multi-view summaries. The storyboard naturally reflects correlations among multi-view summarized shots that describe the same important event. The event-board serially assembles event-centered multi-view shots in temporal order. Single video summary which facilitates quick browsing of the summarized multi-view video can be easily generated based on the event board representation.
Optical Flow-Based Video Sketch Graph Extraction.

Qiyuan Du,Yiping Duan,Zhipeng Xie,Xiaoming Tao,Renxiao Zeng,Zhijuan Jin,Fei Li

DOI: https://doi.org/10.1109/GCWkshps58843.2023.10464524

2023-01-01

Abstract:With the rapid development of video services and the dramatic growth of video data, efficient video representation and compression are urgently needed for multimedia semantic communication systems. Traditional pixel-based video encoding standards suffer from poor performance under limited bandwidth. With the assistance of deep learning and data-driven approaches, semantic communication theory and semantic-oriented video characterization have attracted attention, and video sketches have potential. Existing sketch extraction algorithms suffer from low efficiency and high bandwidth requirements. To address these challenges, this paper proposes spatial-temporal sketch representations based on the optical flows. Learning-based edge detection and motion estimation techniques are integrated to generate sketch sequences for videos. To quantitatively evaluate the performance of the proposed algorithm, experiments were conducted on the Cityscapes dataset. The results demonstrate that the proposed method outperformed existing approaches in terms of processing efficiency and bandwidth requirements, highlighting its potential for video representation in multimedia semantic communication systems.
Learning Effective Geometry Representation from Videos for Self-Supervised Monocular Depth Estimation

Hailiang Zhao,Yongyi Kong,Chonghao Zhang,Haoji Zhang,Jiansen Zhao

DOI: https://doi.org/10.3390/ijgi13060193

IF: 3.4

2024-06-12

ISPRS International Journal of Geo-Information

Abstract:Recent studies on self-supervised monocular depth estimation have achieved promising results, which are mainly based on the joint optimization of depth and pose estimation via high-level photometric loss. However, how to learn the latent and beneficial task-specific geometry representation from videos is still far from being explored. To tackle this issue, we propose two novel schemes to learn more effective representation from monocular videos: (i) an Inter-task Attention Model (IAM) to learn the geometric correlation representation between the depth and pose learning networks to make structure and motion information mutually beneficial; (ii) a Spatial-Temporal Memory Module (STMM) to exploit long-range geometric context representation among consecutive frames both spatially and temporally. Systematic ablation studies are conducted to demonstrate the effectiveness of each component. Evaluations on KITTI show that our method outperforms current state-of-the-art techniques.

geography, physical,remote sensing,computer science, information systems
Video Synopsis Generation Using Spatio-Temporal Groups

A. Ahmed,D. P. Dogra,S. Kar,R. Patnaik,S. Lee,H. Choi,I. Kim

DOI: https://doi.org/10.48550/arXiv.1709.05311

2017-09-16

Abstract:Millions of surveillance cameras operate at 24x7 generating huge amount of visual data for processing. However, retrieval of important activities from such a large data can be time consuming. Thus, researchers are working on finding solutions to present hours of visual data in a compressed, but meaningful way. Video synopsis is one of the ways to represent activities using relatively shorter duration clips. So far, two main approaches have been used by researchers to address this problem, namely synopsis by tracking moving objects and synopsis by clustering moving objects. Synopses outputs, mainly depend on tracking, segmenting, and shifting of moving objects temporally as well as spatially. In many situations, tracking fails, thus produces multiple trajectories of the same object. Due to this, the object may appear and disappear multiple times within the same synopsis output, which is misleading. This also leads to discontinuity and often can be confusing to the viewer of the synopsis. In this paper, we present a new approach for generating compressed video synopsis by grouping tracklets of moving objects. Grouping helps to generate a synopsis where chronologically related objects appear together with meaningful spatio-temporal relation. Our proposed method produces continuous, but a less confusing synopses when tested on publicly available dataset videos as well as in-house dataset videos.

Computer Vision and Pattern Recognition
Structure Analysis For Plane Geometry Figures

Tianxiao Feng,Xiaoqing Lu,Lu Liu,Keqiang Li,Zhi Tang

DOI: https://doi.org/10.1117/12.2042462

2014-01-01

Abstract:As there are increasing numbers of digital documents for education purpose, we realize that there is not a retrieval application for mathematic plane geometry images. In this paper, we propose a method for retrieving plane geometry figures (PGFs), which often appear in geometry books and digital documents. First, detecting algorithms are applied to detect common basic geometry shapes from a PGF image. Based on all basic shapes, we analyze the structural relationships between two basic shapes and combine some of them to a compound shape to build the PGF descriptor. Afterwards, we apply matching function to retrieve candidate PGF images with ranking. The great contribution of the paper is that we propose a structure analysis method to better describe the spatial relationships in such image composed of many overlapped shapes. Experimental results demonstrate that our analysis method and shape descriptor can obtain good retrieval results with relatively high effectiveness and efficiency.
A Sketch-Based Approach for Interactive Organization of Video Clips

Yong-Jin Liu,Cui-Xia Ma,Qiufang Fu,Xiaolan Fu,Sheng-Feng Qin,Lexing Xie

DOI: https://doi.org/10.1145/2645643

2014-01-01

Abstract:With the rapid growth of video resources, techniques for efficient organization of video clips are becoming appealing in the multimedia domain. In this article, a sketch-based approach is proposed to intuitively organize video clips by: (1) enhancing their narrations using sketch annotations and (2) structurizing the organization process by gesture-based free-form sketching on touch devices. There are two main contributions of this work. The first is a sketch graph, a novel representation for the narrative structure of video clips to facilitate content organization. The second is a method to perform context-aware sketch recommendation scalable to large video collections, enabling common users to easily organize sketch annotations. A prototype system integrating the proposed approach was evaluated on the basis of five different aspects concerning its performance and usability. Two sketch searching experiments showed that the proposed context-aware sketch recommendation outperforms, in terms of accuracy and scalability, two state-of-the-art sketch searching methods. Moreover, a user study showed that the sketch graph is consistently preferred over traditional representations such as keywords and keyframes. The second user study showed that the proposed approach is applicable in those scenarios where the video annotator and organizer were the same person. The third user study showed that, for video content organization, using sketch graph users took on average 1/3 less time than using a mass-market tool Movie Maker and took on average 1/4 less time than using a state-of-the-art sketch alternative. These results demonstrated that the proposed sketch graph approach is a promising video organization tool.

Applications of Geometry Processing

Description and Browsing of Video Story Structure

Visual Storylines: Semantic Visualization of Movie Sequence.

An Interactive Personalized Video Summarization Based on Sketches.

Application of Geometry in Plural Space in physics

Compounded Geometry Image Representation of Point Cloud Surfaces and Its Application

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Video Presentation Board : A Semantic Visualization of Video Sequence

Geometry Textures and Applications

An Effective Video Synopsis Approach with Seam Carving

A Depth Extraction Method Based on Motion and Geometry for 2D to 3D Conversion

Automatically Generating Hierarchical Summary for Film Video

Generating fluent tubes in video synopsis

Refilming with Depth-Inferred Videos

Multi-video Summarization Using Complex Graph Clustering and Mining.

Multi-View Video Summarization

Optical Flow-Based Video Sketch Graph Extraction.

Learning Effective Geometry Representation from Videos for Self-Supervised Monocular Depth Estimation

Video Synopsis Generation Using Spatio-Temporal Groups

Structure Analysis For Plane Geometry Figures

A Sketch-Based Approach for Interactive Organization of Video Clips