Abstract:Non-rigid structure-from-motion (NRSfM) has so far been mostly studied for recovering 3D structure of a single non-rigid/deforming object. To handle the real world challenging multiple deforming objects scenarios, existing methods either pre-segment different objects in the scene or treat multiple non-rigid objects as a whole to obtain the 3D non-rigid reconstruction. However, these methods fail to exploit the inherent structure in the problem as the solution of segmentation and the solution of reconstruction could not benefit each other. In this paper, we propose a unified framework to jointly segment and reconstruct multiple non-rigid objects. To compactly represent complex multi-body non-rigid scenes, we propose to exploit the structure of the scenes along both temporal direction and spatial direction, thus achieving a spatio-temporal representation. Specifically, we represent the 3D non-rigid deformations as lying in a union of subspaces along the temporal direction and represent the 3D trajectories as lying in the union of subspaces along the spatial direction. This spatio-temporal representation not only provides competitive 3D reconstruction but also outputs robust segmentation of multiple non-rigid objects. The resultant optimization problem is solved efficiently using the Alternating Direction Method of Multipliers (ADMM). Extensive experimental results on both synthetic and real multi-body NRSfM datasets demonstrate the superior performance of our proposed framework compared with the state-of-the-art methods.

Nonrigid Structure from Motion Via Sparse Representation.

Robust Structure from Motion with Affine Camera Via Low-Rank Matrix Recovery

Dense Non-Rigid Structure from Motion: A Manifold Viewpoint

Dense Non-Rigid Structure-from-motion Made Easy — A Spatial-Temporal Smoothness Based Solution

TC-SfM: Robust Track-Community-Based Structure-from-Motion

3D Shape Estimation With an Enhanced Sparse Representation Approach

Trajectory Space: A Dual Representation for Nonrigid Structure from Motion

3-D Skeleton Recovery via Sparse Representation

Recovering Complex Non-Rigid 3d Structures From Monocular Images By Union Of Nonlinear Subspaces

Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective

Recovery Of Nonrigid Structures From 2d Observations

Nonrigid Structure-From-Motion from 2-D Images Using Markov Chain Monte Carlo.

Monocular 3D Reconstruction of Multiple Non-Rigid Objects by Union of Non-linear Spatial-Temporal Subspaces.

Estimating small structural motions based on sparsity enforcement

Object Tracking by Occlusion Detection via Structured Sparse Learning

Structure from Motion: Sparse Versus Dense Correspondence Methods

Spatio-Temporal Reconstruction for 3D Motion Recovery

Spatio-temporal Union of Subspaces for Multi-Body Non-Rigid Structure-from-motion

Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data

Ordered Subspace Clustering for Complex Non-Rigid Motion by 3D Reconstruction

Structure from Recurrent Motion: from Rigidity to Recurrency