Abstract:Recovering the 3D structure of the scene from images yields useful information for tasks such as shape and scene recognition, object detection, or motion planning and object grasping in robotics. In this thesis, we introduce a general machine learning approach called unsupervised CRF learning based on maximizing the conditional likelihood. We apply our approach to computer vision systems that recover the 3-D scene geometry from images. We focus on recovering 3D geometry from single images, stereo pairs and video sequences. Building these systems requires algorithms for doing inference as well as learning the parameters of conditional Markov random fields (MRF). Our system is trained unsupervisedly without using ground-truth labeled data. We employ a slanted-plane stereo vision model in which we use a fixed over-segmentation to segment the left image into coherent regions called superpixels, then assign a disparity plane for each superpixel. Plane parameters are estimated by solving an MRF labelling problem, through minimizing an energy fuction. We demonstrate the use of our unsupervised CRF learning algorithm for a parameterized slanted-plane stereo vision model involving shape from texture cues. Our stereo model with texture cues, only by unsupervised training, outperforms the results in related work on the same stereo dataset. In this thesis, we also formulate structure and motion estimation as an energy minimization problem, in which the model is an extension of our slanted-plane stereo vision model that also handles surface velocity. Velocity estimation is achieved by solving an MRF labeling problem using Loopy BP. Performance analysis is done using our novel evaluation metrics based on the notion of view prediction error. Experiments on road-driving stereo sequences show encouraging results.

Structure recovery algorithm using contextual information

Structure-Aware Image Completion with Texture Propagation

Im2Struct: Recovering 3D Shape Structure from a Single RGB Image

Outdoor Scene 3D Reconstruction from Multiple Point Cloud

Superpixel Segmentation Based Structural Scene Recognition.

Three-Dimensional Structure Measurement And Optimization Method Of Indoor Scene Based On Single Image

TC-SfM: Robust Track-Community-Based Structure-from-Motion

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Structure Recovery by Part Assembly

Semantic Reconstruction based on RGB Image and Sparse Depth

Structure Preserving Large Imagery Reconstruction

Massive-Scale Visual Information Retrieval towards City Residential Environment Surveillance

Online Structure Analysis for Real-Time Indoor Scene Reconstruction

Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture

Indoor Structure Understanding from Single 360 Cylindrical Panoramic Image

Complex spatial region representation and similar matching for multi-object image retrieval

A new Image Decomposition Approach using Pixel-wise Analysis Sparsity Model

Recovering 3D Planar Arrangements from Videos

A Machine Learning Approach to Recovery of Scene Geometry from Images

Exploiting Object Similarity in 3D Reconstruction

A Unified Framework for Piecewise Semantic Reconstruction in Dynamic Scenes Via Exploiting Superpixel Relations.