Minimal Problems for the Calibrated Trifocal Variety

Joe Kileel

DOI: https://doi.org/10.48550/arXiv.1611.05947

2016-11-18

Abstract:We determine the algebraic degree of minimal problems for the calibrated trifocal variety in computer vision. We rely on numerical algebraic geometry and the homotopy continuation software Bertini.

Algebraic Geometry,Computer Vision and Pattern Recognition,Numerical Analysis

What problem does this paper attempt to address?

Calibrated geometry in hyperkähler cones, 3-Sasakian manifolds, and twistor spaces

Benjamin Aslan,Spiro Karigiannis,Jesse Madnick

DOI: https://doi.org/10.4153/s0008414x24000282

2024-04-01

Canadian Journal of Mathematics

Abstract:Abstract We systematically study calibrated geometry in hyperkähler cones $C^{4n+4}$ , their 3-Sasakian links $M^{4n+3}$ , and the corresponding twistor spaces $Z^{4n+2}$ , emphasizing the relationships between submanifold geometries in various spaces. Our analysis highlights the role played by a canonical $\mathrm {Sp}(n)\mathrm {U}(1)$ -structure $\gamma $ on the twistor space Z . We observe that $\mathrm {Re}(e^{- i \theta } \gamma )$ is an $S^1$ -family of semi-calibrations and make a detailed study of their associated calibrated geometries. As an application, we obtain new characterizations of complex Lagrangian and complex isotropic cones in hyperkähler cones, generalizing a result of Ejiri–Tsukada. We also generalize a theorem of Storm on submanifolds of twistor spaces that are Lagrangian with respect to both the Kähler–Einstein and nearly Kähler structures.

mathematics
Minimal Perspective Autocalibration

Andrea Porfiri Dal Cin,Timothy Duff,Luca Magri,Tomas Pajdla

2024-05-09

Abstract:We introduce a new family of minimal problems for reconstruction from multiple views. Our primary focus is a novel approach to autocalibration, a long-standing problem in computer vision. Traditional approaches to this problem, such as those based on Kruppa's equations or the modulus constraint, rely explicitly on the knowledge of multiple fundamental matrices or a projective reconstruction. In contrast, we consider a novel formulation involving constraints on image points, the unknown depths of 3D points, and a partially specified calibration matrix $K$. For $2$ and $3$ views, we present a comprehensive taxonomy of minimal autocalibration problems obtained by relaxing some of these constraints. These problems are organized into classes according to the number of views and any assumed prior knowledge of $K$. Within each class, we determine problems with the fewest -- or a relatively small number of -- solutions. From this zoo of problems, we devise three practical solvers. Experiments with synthetic and real data and interfacing our solvers with COLMAP demonstrate that we achieve superior accuracy compared to state-of-the-art calibration methods. The code is available at

Computer Vision and Pattern Recognition
Problems and related results in Algebraic Vision and Multiview Geometry

Marina Bertolini,Cristina Turrini

2024-01-31

Abstract:The article presents a survey of results in algebraic vision and multiview geometry. The starting points is the study of projective algebraic varieties critical for scene reconstruction. Initially studied for reconstructing static scenes in three-dimensional spaces, these critical loci are later investigated for dynamic and segmented scenes in higher-dimensional projective spaces. The formal analysis of the ideals of critical loci employs Grassmann tensors, introduced as crucial tools for determining these ideals and aiding the reconstruction process away from critical loci. A long-term goal of the authors with other co-authors involves two main aspects: firstly studying properties of Grassmann tensors, as rank, multi-rank and core, along with the varieties that parameterize these tensors; concurrently conducting an analysis of families of critical loci in various scenarios.

Algebraic Geometry
Line Multiview Ideals

Paul Breiding,Timothy Duff,Lukas Gustafsson,Felix Rydell,Elima Shehu

2024-04-23

Abstract:We study the following problem in computer vision from the perspective of algebraic geometry: Using $m$ pinhole cameras we take $m$ pictures of a line in $\mathbb P^3$. This produces $m$ lines in $\mathbb P^2$ and the question is which $m$-tuples of lines can arise that way. We are interested in polynomial equations and therefore study the complex Zariski closure of all such tuples of lines. The resulting algebraic variety is a subvariety of $(\mathbb P^2)^m$ and is called line multiview variety. In this article, we study its ideal. We show that for generic cameras the ideal is generated by $3\times 3$-minors of a specific matrix. We also compute Gröbner bases and discuss to what extent our results carry over to the non-generic case.

Algebraic Geometry
PLMP -- Point-Line Minimal Problems in Complete Multi-View Visibility

Timothy Duff,Kathlén Kohn,Anton Leykin,Tomas Pajdla

DOI: https://doi.org/10.48550/arXiv.1903.10008

2019-09-05

Abstract:We present a complete classification of all minimal problems for generic arrangements of points and lines completely observed by calibrated perspective cameras. We show that there are only 30 minimal problems in total, no problems exist for more than 6 cameras, for more than 5 points, and for more than 6 lines. We present a sequence of tests for detecting minimality starting with counting degrees of freedom and ending with full symbolic and numeric verification of representative examples. For all minimal problems discovered, we present their algebraic degrees, i.e. the number of solutions, which measure their intrinsic difficulty. It shows how exactly the difficulty of problems grows with the number of views. Importantly, several new minimal problems have small degrees that might be practical in image matching and 3D reconstruction.

Computer Vision and Pattern Recognition,Algebraic Geometry,Numerical Analysis
Average degree of the essential variety

Paul Breiding,Samantha Fairchild,Pierpaola Santarsiero,Elima Shehu

2023-11-10

Abstract:The essential variety is an algebraic subvariety of dimension $5$ in real projective space $\mathbb R\mathrm P^{8}$ which encodes the relative pose of two calibrated pinhole cameras. The $5$-point algorithm in computer vision computes the real points in the intersection of the essential variety with a linear space of codimension $5$. The degree of the essential variety is $10$, so this intersection consists of 10 complex points in general. We compute the expected number of real intersection points when the linear space is random. We focus on two probability distributions for linear spaces. The first distribution is invariant under the action of the orthogonal group $\mathrm{O}(9)$ acting on linear spaces in $\mathbb R\mathrm P^{8}$. In this case, the expected number of real intersection points is equal to $4$. The second distribution is motivated from computer vision and is defined by choosing 5 point correspondences in the image planes $\mathbb R\mathrm P^2\times \mathbb R\mathrm P^2$ uniformly at random. A Monte Carlo computation suggests that with high probability the expected value lies in the interval $(3.95 - 0.05,\ 3.95 + 0.05)$.

Algebraic Geometry,Computer Vision and Pattern Recognition,Geometric Topology
Algebra and Geometry of Camera Resectioning

Erin Connelly,Timothy Duff,Jessie Loucks-Tavitas

2023-09-08

Abstract:We study algebraic varieties associated with the camera resectioning problem. We characterize these resectioning varieties' multigraded vanishing ideals using Gröbner basis techniques. As an application, we derive and re-interpret celebrated results in geometric computer vision related to camera-point duality. We also clarify some relationships between the classical problems of optimal resectioning and triangulation, state a conjectural formula for the Euclidean distance degree of the resectioning variety, and discuss how this conjecture relates to the recently-resolved multiview conjecture.

Algebraic Geometry,Computer Vision and Pattern Recognition,Commutative Algebra
Trifocal Relative Pose from Lines at Points and its Efficient Solution

Ricardo Fabbri,Timothy Duff,Hongyi Fan,Margaret Regan,David da Costa de Pinho,Elias Tsigaridas,Charles Wampler,Jonathan Hauenstein,Benjamin Kimia,Anton Leykin,Tomas Pajdla

DOI: https://doi.org/10.48550/arXiv.1903.09755

2022-11-30

Abstract:We present a method for solving two minimal problems for relative camera pose estimation from three views, which are based on three view correspondences of i) three points and one line and the novel case of ii) three points and two lines through two of the points. These problems are too difficult to be efficiently solved by the state of the art Groebner basis methods. Our method is based on a new efficient homotopy continuation (HC) solver framework MINUS, which dramatically speeds up previous HC solving by specializing HC methods to generic cases of our problems. We characterize their number of solutions and show with simulated experiments that our solvers are numerically robust and stable under image noise, a key contribution given the borderline intractable degree of nonlinearity of trinocular constraints. We show in real experiments that i) SIFT feature location and orientation provide good enough point-and-line correspondences for three-view reconstruction and ii) that we can solve difficult cases with too few or too noisy tentative matches, where the state of the art structure from motion initialization fails.

Computer Vision and Pattern Recognition
The Kodaira-Spencer map for minimal toric hypersurfaces

Julius Giesler

2023-11-18

Abstract:In this article we study infinitesimal deformations of toric hypersurfaces. We introduce a Kodaira-Spencer map and compute its kernel. By introducing some new Laurent polynomials we make our computation as explicit as possible. This widely generalizes results of Griffiths for projective hypersurfaces.

Algebraic Geometry
Birational geometry of critical loci in Algebraic Vision

Marina Bertolini,Roberto Notari,Cristina Turrini

2024-08-04

Abstract:In Algebraic Vision, the projective reconstruction of the position of each camera and scene point from the knowledge of many enough corresponding points in the views is called the structure from motion problem. It is known that the reconstruction is ambiguous if the scene points are contained in particular algebraic varieties, called critical loci. To be more precise, from the definition of criticality, for the same reconstruction problem, two critical loci arise in a natural way. In the present paper, we investigate the relations between these two critical loci, and we prove that, under some mild smoothness hypotheses, (some of) their irreducible components are birational. To this end, we introduce a unified critical locus that restores the symmetry between the two critical loci, and a natural commutative diagram relating the unified critical locus and the two single critical loci. For technical reasons, but of interest in its own, we also consider how a critical locus change when one increases the number of views.

Algebraic Geometry
Extrinsic geometry of calibrated submanifolds

Karigiannis, Spiro,Martín-Merchán, Lucía

DOI: https://doi.org/10.1007/s00209-024-03503-x

2024-05-14

Mathematische Zeitschrift

Abstract:Given a calibration whose stabilizer acts transitively on the Grassmanian of calibrated planes, we introduce a nontrivial Lie-theoretic condition on , which we call compliancy , and show that this condition holds for many interesting geometric calibrations, including Kähler, special Lagrangian, associative, coassociative, and Cayley. We determine a sufficient condition that ensures compliancy of , we completely characterize compliancy in terms of properties of a natural involution determined by a calibrated plane, and we relate compliancy to the geometry of the calibrated Grassmanian. The condition that a Riemannian immersion be calibrated is a first order condition. By contrast, its extrinsic geometry, given by the second fundamental form A and the induced tangent and normal connections on TL and D on NL , respectively, is second order information. We characterize the conditions imposed on the extrinsic geometric data when the Riemannian immersion is calibrated with respect to a calibration on M which is both parallel and compliant . This motivate the definition of an infinitesimally calibrated Riemannian immersion, generalizing the classical notion of a superminimal surface in .

mathematics
Centering toric arrangements of maximal rank

Elia Saini

2024-10-29

Abstract:The homotopy type of the complement manifold of a complexified toric arrangement has been investigated by d'Antonio and Delucchi in a paper that shows the minimality of such topological space. In this work we associate to a given toric arrangement a matrix that represents the arrangement over the integers. Then, we consider the family of toric arrangements for which this matrix has maximal rank. Our goal is to prove, by means of basic linear algebra arguments, that the complement manifold of the toric arrangements that belong to this family is diffeomorphic to that of centered toric arrangements and thus it is a minimal topological space, too.

Combinatorics,Algebraic Topology,Geometric Topology
Projections of Curves and Conic Multiview Varieties

Felix Rydell,Isak Sundelius

2024-04-04

Abstract:We present an algebraic study of the projection of plane curves and twisted cubics in space onto multiple images of pinhole cameras. The Zariski closure of the image of the projection of conics is a conic multiview varieties. Extending previous work for point and line multiview varieties, we use back-projected cones to describe these varieties. For two views, we provide the defining ideal of the multiview variety. For any number of views, we state when the simplest possible set-theoretic description is achieved based on the geometry of the camera centers. Finally, we investigate the complexity of the associated triangulation problem and conjecture the Euclidean distance degree for the conic multiview variety for two cameras.

Algebraic Geometry
The Multilinear Rank and Core of Trifocal Grassmann Tensors

Marina Bertolini,Gian Mario Besana,Gilberto Bini,Cristina Turrini

2023-12-04

Abstract:Closed formulas for the multilinear rank of trifocal Grassmann tensors are obtained. An alternative process to the standard HOSVD is introduced for the computation of the core of trifocal Grassmann tensors. Both of these results are obtained, under natural genericity conditions, leveraging the canonical form for these tensors, obtained by the same authors in a previous work. A gallery of explicit examples is also included.

Algebraic Geometry
Calibrated and Partially Calibrated Semi-Generalized Homographies

Snehal Bhayani,Torsten Sattler,Daniel Barath,Patrik Beliansky,Janne Heikkila,Zuzana Kukelova

DOI: https://doi.org/10.48550/arXiv.2103.06535

2021-10-11

Abstract:In this paper, we propose the first minimal solutions for estimating the semi-generalized homography given a perspective and a generalized camera. The proposed solvers use five 2D-2D image point correspondences induced by a scene plane. One of them assumes the perspective camera to be fully calibrated, while the other solver estimates the unknown focal length together with the absolute pose parameters. This setup is particularly important in structure-from-motion and image-based localization pipelines, where a new camera is localized in each step with respect to a set of known cameras and 2D-3D correspondences might not be available. As a consequence of a clever parametrization and the elimination ideal method, our approach only needs to solve a univariate polynomial of degree five or three. The proposed solvers are stable and efficient as demonstrated by a number of synthetic and real-world experiments.

Computer Vision and Pattern Recognition
PL P: Point-Line Minimal Problems under Partial Visibility in Three Views

Timothy Duff,Kathlén Kohn,Anton Leykin,Tomas Pajdla

DOI: https://doi.org/10.1007/s11263-024-01992-1

IF: 13.369

2024-03-12

International Journal of Computer Vision

Abstract:We present a complete classification of minimal problems for generic arrangements of points and lines in space observed partially by three calibrated perspective cameras when each line is incident to at most one point. This is a large class of interesting minimal problems that allows missing observations in images due to occlusions and missed detections. There is an infinite number of such minimal problems; however, we show that they can be reduced to 140,616 equivalence classes by removing superfluous features and relabeling the cameras. We also introduce camera-minimal problems, which are practical for designing minimal solvers, and show how to pick a simplest camera-minimal problem for each minimal problem. This simplification results in 74,575 equivalence classes. Only 76 of these were known; the rest are new. To identify problems having potential for practical solving of image matching and 3D reconstruction, we present several natural subfamilies of camera-minimal problems as well as compute solution counts for all camera-minimal problems which have fewer than 300 solutions for generic data.

computer science, artificial intelligence
Projections of Higher Dimensional Subspaces and Generalized Multiview Varieties

Felix Rydell

2024-04-04

Abstract:We present a generalization of multiview varieties as closures of images obtained by projecting subspaces of a given dimension onto several views, from the photographic and geometric points of view. Motivated by applications in Computer Vision for triangulation of world features, we investigate when the associated projection map is generically injective; an essential requirement for successful triangulation. We give a complete characterization of this property by determining two formulae for the dimensions of these varieties. Similarly, we describe for which center arrangements calibration of camera parameters is possible. We explore when the multiview variety is naturally isomorphic to its associated blowup. In the case of generic centers, we give a precise formula for when this occurs.

Algebraic Geometry
Lines, Quadrics, and Cremona Transformations in Two-View Geometry

Erin Connelly,Rekha R. Thomas,Cynthia Vinzant

2024-03-19

Abstract:Given $7 \leq k \leq 9$ points $(x_i,y_i) \in \mathbb{P}^2 \times \mathbb{P}^2$, we characterize rank deficiency of the $k \times 9$ matrix $Z_k$ with rows $x_i^\top \otimes y_i^\top$, in terms of the geometry of the point sets $\{x_i\}$ and $\{y_i\}$. This problem arises in the conditioning of certain well-known reconstruction algorithms in computer vision, but has surprising connections to classical algebraic geometry via the interplay of quadric surfaces, cubic curves and Cremona transformations. The characterization of rank deficiency of $Z_k$, when $k \leq 6$, was completed in <a class="link-https" data-arxiv-id="2301.09826" href="https://arxiv.org/abs/2301.09826">arXiv:2301.09826</a>.

Algebraic Geometry,Commutative Algebra,Combinatorics

Minimal Problems for the Calibrated Trifocal Variety

Calibrated geometry in hyperkähler cones, 3-Sasakian manifolds, and twistor spaces

Minimal Perspective Autocalibration

Problems and related results in Algebraic Vision and Multiview Geometry

Line Multiview Ideals

PLMP -- Point-Line Minimal Problems in Complete Multi-View Visibility

Average degree of the essential variety

Algebra and Geometry of Camera Resectioning

Trifocal Relative Pose from Lines at Points and its Efficient Solution

The Kodaira-Spencer map for minimal toric hypersurfaces

Birational geometry of critical loci in Algebraic Vision

Extrinsic geometry of calibrated submanifolds

Centering toric arrangements of maximal rank

Projections of Curves and Conic Multiview Varieties

The Multilinear Rank and Core of Trifocal Grassmann Tensors

Calibrated and Partially Calibrated Semi-Generalized Homographies

PL P: Point-Line Minimal Problems under Partial Visibility in Three Views

Projections of Higher Dimensional Subspaces and Generalized Multiview Varieties

Lines, Quadrics, and Cremona Transformations in Two-View Geometry