Abstract:The field of multiple view geometry has seen tremendous progress in reconstruction and calibration due to methods for extracting reliable point features and key developments in projective geometry. Point features, however, are not available in certain applications and result in unstructured point cloud reconstructions. General image curves provide a complementary feature when keypoints are scarce, and result in 3D curve geometry, but face challenges not addressed by the usual projective geometry of points and algebraic curves. We address these challenges by laying the theoretical foundations of a framework based on the differential geometry of general curves, including stationary curves, occluding contours, and non-rigid curves, aiming at stereo correspondence, camera estimation (including calibration, pose, and multiview epipolar geometry), and 3D reconstruction given measured image curves. By gathering previous results into a cohesive theory, novel results were made possible, yielding three contributions. First, we derive the differential geometry of an image curve (tangent, curvature, curvature derivative) from that of the underlying space curve (tangent, curvature, curvature derivative, torsion). Second, we derive the differential geometry of a space curve from that of two corresponding image curves. Third, the differential motion of an image curve is derived from camera motion and the differential geometry and motion of the space curve. The availability of such a theory enables novel curve-based multiview reconstruction and camera estimation systems to augment existing point-based approaches. This theory has been used to reconstruct a “3D curve sketch”, to determine camera pose from local curve geometry, and tracking; other developments are underway.

From Multiview Image Curves to 3D Drawings

The Surfacing of Multiview 3D Drawings via Lofting and Occlusion Reasoning

Multiview Differential Geometry of Curves

Sketch-based 3D Modeling by Aligning Outlines of an Image

An Example-Based Approach to 3D Man-Made Object Reconstruction from Line Drawings

Sketch-based 3-D modeling for piecewise planar objects in single images

Embedding Visual Cognition in 3D Reconstruction from Multi-View Engineering Drawings

Model-driven sketch reconstruction with structure-oriented retrieval

Precise 3d reconstruction from a single image

Reconstruction Of 3-D Symmetric Curves From Perspective Images Without Discrete Features

Vid2Curve

Diff3DS: Generating View-Consistent 3D Sketch via Differentiable Curve Rendering

Reconstruction of Curved Solids from Engineering Drawings

Curved Image Triangulation Based on Differentiable Rendering

Reconstructing Curves from Sparse Samples on Riemannian Manifolds

Reconstruction of 3D Curvilinear Wire-Frame from Three Orthographic Views

Efficient Depth Restoration from 2D Line Drawings with Line Segments and Curves

Vid2Curve: simultaneous camera motion estimation and thin structure reconstruction from an RGB video

Data-Driven Space-Filling Curves

MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images

3-D Reconstruction of Three Views Based on Manifold Study