Abstract:The field of multiple view geometry has seen tremendous progress in reconstruction and calibration due to methods for extracting reliable point features and key developments in projective geometry. Point features, however, are not available in certain applications and result in unstructured point cloud reconstructions. General image curves provide a complementary feature when keypoints are scarce, and result in 3D curve geometry, but face challenges not addressed by the usual projective geometry of points and algebraic curves. We address these challenges by laying the theoretical foundations of a framework based on the differential geometry of general curves, including stationary curves, occluding contours, and non-rigid curves, aiming at stereo correspondence, camera estimation (including calibration, pose, and multiview epipolar geometry), and 3D reconstruction given measured image curves. By gathering previous results into a cohesive theory, novel results were made possible, yielding three contributions. First, we derive the differential geometry of an image curve (tangent, curvature, curvature derivative) from that of the underlying space curve (tangent, curvature, curvature derivative, torsion). Second, we derive the differential geometry of a space curve from that of two corresponding image curves. Third, the differential motion of an image curve is derived from camera motion and the differential geometry and motion of the space curve. The availability of such a theory enables novel curve-based multiview reconstruction and camera estimation systems to augment existing point-based approaches. This theory has been used to reconstruct a “3D curve sketch”, to determine camera pose from local curve geometry, and tracking; other developments are underway.

Multiple-view geometry for image-based modeling.

Multi-view 3D Reconstruction from Uncalibrated Radially-Symmetric Cameras

An Invitation to 3-D Vision: From Images to Geometric Models

Multi-View Stereo: A Tutorial

View-Aware Geometry-Structure Joint Learning for Single-View 3D Shape Reconstruction

Multi-View Geometry Compression

A Lie Theoretic Approach to Structure and Motion in Computer Vision 1

A Survey of Geometric Vision

Geometric Principles of Multiple Visual Sensors

Generalized Rank Conditions In Multiple View Geometry With Applications To Dynamical Scenes

An Interactive Image-Based Modeling Approach With Scene Constraints

Problems and related results in Algebraic Vision and Multiview Geometry

Multiview Differential Geometry of Curves

Multi-view 3D reconstruction and modeling of the unknown 3D scenes using genetic algorithms

On Symmetry and Multiple-View Geometry: Structure, Pose, and Calibration from a Single Image

A Differential Geometric Approach to Multiple View Geometry in Spaces of Constant Curvature

Cameras Viewing Cameras Geometry

Multi-view coding for image-based rendering using 3-d scene geometry

Rank Conditions on the Multiple-View Matrix

Graph-based representation for multiview image geometry

GC-MVSNet: Multi-View, Multi-Scale, Geometrically-Consistent Multi-View Stereo