Abstract:Multiple view geometry is the foundation of an important class of computer vision techniques for simultaneous recovery of camera motion and scene structure from a set of images. There are numerous important applications in this area. Examples include video post-production, scene reconstruction, registration, surveillance, tracking, and segmentation. In video post-production, which is the topic being addressed in this dissertation, computer analysis of the motion of the camera can replace the currently used manual methods for correctly aligning an artificially inserted object in a scene. However, existing single view methods typically require multiple vanishing points, and therefore would fail when only one vanishing point is available. In addition, current multiple view techniques, making use of either epipolar geometry or trifocal tensor, do not exploit fully the properties of constant or known camera motion. Finally, there does not exist a general solution to the problem of synchronization of N video sequences of distinct general scenes captured by cameras undergoing similar ego-motions, which is the necessary step for video post-production among different input videos. This dissertation proposes several advancements that overcome these limitations. These advancements are used to develop an efficient framework for video analysis and post-production in multiple cameras. In the first part of the dissertation, the novel inter-image constraints are introduced that are particularly useful for scenes where minimal information is available. This result extends the current state-of-the-art in single view geometry technique to situations where only one vanishing point is available. The property of constant or known camera motion is also described in this dissertation for applications such as calibration of a network of cameras in video surveillance systems, and Euclidean reconstruction from turn-table image sequences in the presence of zoom and focus. We then propose a new framework for the estimation and alignment of camera motions, including both simple (panning, tracking and zooming) and complex (e.g. hand-held) camera motions. Accuracy of these results is demonstrated by applying our approach to video post-production applications such as video cut-and-paste and shadow synthesis. As realistic image-based rendering problems, these applications require extreme accuracy in the estimation of camera geometry, the position and the orientation of the light source, and the photometric properties of the resulting cast shadows. In each case, the theoretical results are fully supported and illustrated by both numerical simulations and thorough experimentation on real data.

A Lie Theoretic Approach to Structure and Motion in Computer Vision 1

Multiple view geometry for video analysis and post-production

Structure and Pose from Single Images of Symmetric Objects with Applications to Robot Navigation

Invariant Cubature Kalman Filtering-Based Visual-Inertial Odometry for Robot Pose Estimation

Multiple-view geometry for image-based modeling.

Cameras Viewing Cameras Geometry

Two-View Multibody Structure from Motion

A micro Lie theory for state estimation in robotics

Motion And Structure From An Orthographic And A Perspective Image

Some Aspects of Geometric Computer Vision for Analysing Dynamical Scenes focusing Automotive Applications

A Survey of Geometric Vision

Euclidean Structure and Motion From Image Sequences

A New Paradigm for Dealing With Manifold Structures in Visual Inertial Odometry by Using Stable Embedding

An Invitation to 3-D Vision: From Images to Geometric Models

Motion and Structure from One Perspective and One Orthographic Views

Geometric Principles of Multiple Visual Sensors

Vision Guided Navigation for a Nonholonomic Mobile Robot

Perspective n-view multibody structure-and-motion through model selection

Several camera geometry models and error analysis for image matching in 3-D machine vision

3D Articulated Structure and Motion Analysis from Monocular Images

On Symmetry and Multiple-View Geometry: Structure, Pose, and Calibration from a Single Image