Abstract:This thesis presents a novel method for reconstructing full dynamic environments in real-time using either live or recorded data. The ability to recreate physical environments virtually is a key technology for various applications, and in the last decade we have seen a significant improvement in real-time approaches to geometric scene reconstruction. This began with the development of affordable RGBD cameras (such as the Microsoft Kinect), and the release of KinectFusion in 2011. The case of realtime full scene dynamic reconstruction, however, has remained an open problem. Existing attempts are limited by cumbersome equipment setups or restrictive use cases.Our research is split into two major contributions, which address many limitations of existing approaches to the problem. Our first major contribution is the development of a novel dynamic scene reconstruction method. We use a heterogeneous camera setup with two centralised 360° cameras (one RGB, one depth) and a moving standard field of view RGBD camera. We implemented a working physical prototype of this method, acting as a proof of concept. Our second major contribution is the development of a visual tracking bridging algorithm. With this, we significantly improve the accuracy of existing visual object tracking algorithms on spherical 360° input, enabling their use in our heterogenous camera setup.Our two systems were each evaluated quantitatively. Our dynamic reconstruction method was evaluated on both a synthetic and a real dataset. The synthetic dataset covers three test categories, one static and two dynamic; each made up of data from multiple environments. All evaluations on the synthetic dataset were performed against the current state-of-theart scene reconstruction algorithm. Our system performed comparably in ‘reconstruction completeness’ and slightly better in ‘reconstruction accuracy’ in all static tests - while achieving significantly better results in our dynamic evaluations.We also evaluated our physical prototype in a real world static environment. The system was tested for reconstruction accuracy and completeness against a photogrammetry model which acted as ground truth. In this test, our method did not perform quite as well as it did in the synthetic evaluations, but still achieved a similarly high degree of reconstruction accuracy and reconstruction completeness.Our visual object tracking bridging solution was evaluated on a synthetic equirectangular dataset, where multiple tracking algorithms were tested with and without the assistance of our algorithm to determine its impact on the overall quality of tracking. In these evaluations, our method significantly improved the tracking accuracy and robustness of all the tracking algorithms on each of the datasets.

Real-time 3d reconstruction of dynamic scenes with multiple kinect v2 sensors

RGBD-based Real-time Volumetric Reconstruction System: Architecture Design and Implementation

Robust 3D Reconstruction with an RGB-D Camera

3D Reconstruction of Indoor Scenes Via Image Registration

Real-Time Dynamic Full Scene Reconstruction Using a Heterogeneous Sensor System

A Real-Time Interactive System of Surface Reconstruction and Dynamic Projection Mapping with RGB-depth Sensor and Projector.

Real-time Indoor Scene Reconstruction with RGBD and Inertial Input.

Real-Time 3D Reconstruction Method Based on Monocular Vision

Real-time 3D human objects rendering based on multiple camera details

FastFusion: Real-Time Indoor Scene Reconstruction with Fast Sensor Motion

3D-object reconstruction and point cloud sculpture based on Kinect

3DFusion, A real-time 3D object reconstruction pipeline based on streamed instance segmented data

Mobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone

Visual Odometry Based 3D-Reconstruction

Octree-based Fusion for Realtime 3D Reconstruction

Real-time Depth Extraction and Multi-View Rendering Algorithm Based on Kinect

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Mobile3DScanner: an Online 3D Scanner for High-quality Object Reconstruction with a Mobile Device

Online Structure Analysis for Real-Time Indoor Scene Reconstruction

Online Global Non-rigid Registration for 3D Object Reconstruction Using Consumer-level Depth Cameras

A real-time system for 3D recovery of dynamic scene with multiple RGBD imagers