Abstract:In this paper, we provide a principled explanation of how knowledge in global 3-D structural invariants, typically captured by a group action on a symmetric structure, can dramatically facilitate the task of reconstructing a 3-D scene from one or more images. More importantly, since every symmetric structure admits a “canonical” coordinate frame with respect to which the group action can be naturally represented, the canonical pose between the viewer and this canonical frame can be recovered too, which explains why symmetric objects (e.g., buildings) provide us overwhelming clues to their orientation and position. We give the necessary and sufficient conditions in terms of the symmetry (group) admitted by a structure under which this pose can be uniquely determined. We also characterize, when such conditions are not satisfied, to what extent this pose can be recovered. We show how algorithms from conventional multiple-view geometry, after properly modified and extended, can be directly applied to perform such recovery, from all “hidden images” of one image of the symmetric structure. We also apply our results to a wide range of applications in computer vision and image processing such as camera self-calibration, image segmentation and global orientation, large baseline feature matching, image rendering and photo editing, as well as visual illusions (caused by symmetry if incorrectly assumed).

A Classification of Critical Configurations for any Number of Projective Views

Critical configurations for two projective views, a new approach

Critical configurations for three projective views

Multi-view 3D Reconstruction from Uncalibrated Radially-Symmetric Cameras

Coupled Structure-from-motion and 3D Symmetry Detection for Urban Facades

Birational geometry of critical loci in Algebraic Vision

Euclidean Reconstruction and Reprojection Up to Subgroups

Projections of Higher Dimensional Subspaces and Generalized Multiview Varieties

Projections of Curves and Conic Multiview Varieties

Generalized Rank Conditions In Multiple View Geometry With Applications To Dynamical Scenes

Problems and related results in Algebraic Vision and Multiview Geometry

A structure from motion inequality

Success Probability in Multi-View Imaging

Lines, Quadrics, and Cremona Transformations in Two-View Geometry

On Symmetry and Multiple-View Geometry: Structure, Pose, and Calibration from a Single Image

Projective rigidity of point-line configurations in the plane

Geometric Interpretations of Compatibility for Fundamental Matrices

RESFM: Robust Equivariant Multiview Structure from Motion

Euclidean Structure and Motion From Image Sequences

A Lie Theoretic Approach to Structure and Motion in Computer Vision 1