Abstract:3D face reconstruction from multi-view video sequences has become a hotspot in computer vision for the last decades. Structure from Motion (SfM) methods, which have been widely used for multi-view 3D face reconstruction, have two main limitations. First, self-occlusion causes certain facial feature points (FFPs) to be invisible in the images, which will lead to missing data. The existing SfM methods could recover the missing data through iterative calculation, however, with high computational costs and long processing time. Second, the SfM methods cannot reconstruct the accurate 3D facial shapes of cheeks because there are no FFPs in this area. This paper proposes a novel “coarse-to-fine” multi-view 3D face reconstruction method by taking the advantage of the complementarity between FFPs and occluding contours, i.e., the boundary lines depicted between the facial region and the background. In this method, a block SfM algorithm is firstly proposed to reconstruct a “coarse” 3D facial shape by utilizing sparse FFPs. The block SfM algorithm does not estimate the true locations of the self-occluded FFPs iteratively. Thus, the computational cost is significantly reduced. Then, a kernel partial least squares (KPLS) algorithm is introduced to refine the “coarse” 3D facial shape. The KPLS method applies occluding contours to remedy the limitation of sparse FFPs correspondence-based SfM method. The proposed method is evaluated on the synthetic sequences generated from the BJUT-3D face database and the real-world multi-view video sequences obtained in a controlled indoor environment. The results show improvements in both accuracy and efficiency.

A Keyframe Extraction Approach for 3D Videogrammetry Based on Baseline Constraints

New Fusional Framework Combining Sparse Selection and Clustering for Key Frame Extraction.

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

CRF-Based Reconstruction from Narrow-Baseline Image Sequences.

Local Optimized and Scalable Frame-to-model SLAM

Robust 3D Reconstruction with an RGB-D Camera

Robust Keyframe-based Dense SLAM with an RGB-D Camera.

Vision-Based Scattering Key-Frame Extraction for VideoSAR Summarization

Depth-Guided Sparse Structure-from-Motion for Movies and TV Shows

Robust Camera Motion Estimation in Video Sequences

Key frame extraction from motion capture data based on gpu

A novel video abstraction method based on fast clustering of the regions of interest in key frames

Coarse-to-fine Multiview 3d Face Reconstruction Using Multiple Geometrical Features.

An Efficient Keyframe Extraction from Motion Capture Data

Vid2Curve: simultaneous camera motion estimation and thin structure reconstruction from an RGB video

An efficient 2D to 3D video conversion method based on skeleton line tracking

Keyframe-based RGB-D dense visual SLAM fused semantic cues in dynamic scenes

3D Reconstruction of Indoor Scenes Based on Feature and Graph Optimization

Optimization-based key frame extraction for motion capture animation

Video Key Frame Extraction by Unsupervised Clustering and Feedback Adjustment