Abstract:Artificial intelligence including deep learning and 3D reconstruction methods is changing the daily life of people. Now, an unmanned aerial vehicle that can move freely in the air and avoid harsh ground conditions has been commonly adopted as a suitable tool for 3D reconstruction. The traditional 3D reconstruction mission based on drones usually consists of two steps: image collection and offline post-processing. But there are two problems: one is the uncertainty of whether all parts of the target object are covered, and another is the tedious post-processing time. Inspired by modern deep learning methods, we build a telexistence drone system with an onboard deep learning computation module and a wireless data transmission module that perform incremental real-time dense reconstruction of urban cities by itself. Two technical contributions are proposed to solve the preceding issues. First, based on the popular depth fusion surface reconstruction framework, we combine it with a visual-inertial odometry estimator that integrates the inertial measurement unit and allows for robust camera tracking as well as high-accuracy online 3D scan. Second, the capability of real-time 3D reconstruction enables a new rendering technique that can visualize the reconstructed geometry of the target as navigation guidance in the HMD. Therefore, it turns the traditional path-planning-based modeling process into an interactive one, leading to a higher level of scan completeness. The experiments in the simulation system and our real prototype demonstrate an improved quality of the 3D model using our artificial intelligence leveraged drone system.

FlyFusion: Realtime Dynamic Scene Reconstruction Using a Flying Depth Camera

Adaptive Switching Spatial-Temporal Fusion Detection for Remote Flying Drones

FastFusion: Real-Time Indoor Scene Reconstruction with Fast Sensor Motion

OwlFusion: Depth-Only Onboard Real-Time 3D Reconstruction of Scalable Scenes for Fast-Moving MAV

MixedFusion: Real-Time Reconstruction of an Indoor Scene with Dynamic Objects.

Bodyfusion: Real-Time Capture Of Human Motion And Surface Geometry Using A Single Depth Camera

Dynamic scenes reconstruction based on foreground and background splitting

RigidFusion: RGB‐D Scene Reconstruction with Rigidly‐moving Objects

HeteroFusion: Dense Scene Reconstruction Integrating Multi-Sensors.

Virtual-Real Fusion with Dynamic Scene from Videos

DoubleFusion: Real-Time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor

UnstructuredFusion: Realtime 4D Geometry and Texture Reconstruction Using Commercial RGBD Cameras.

RoutedFusion: Learning Real-time Depth Map Fusion

Scale-Consistent Fusion: From Heterogeneous Local Sampling to Global Immersive Rendering

Idfusion

VisFusion: Visibility-aware Online 3D Scene Reconstruction from Videos

PlaneFusion: Real-Time Indoor Scene Reconstruction With Planar Prior

HDR-Net-Fusion: Real-time 3D dynamic scene reconstruction with a hierarchical deep reinforcement network

Virtual Reality Aided High-Quality 3D Reconstruction by Remote Drones

LF-fusion: Dense and Accurate 3D Reconstruction from Light Field Images

FusionSense: Bridging Common Sense, Vision, and Touch for Robust Sparse-View Reconstruction