Abstract:We present a novel online 3D scanning system for high-quality object reconstruction with a mobile device, called Mobile3DScanner. Using a mobile device equipped with an embedded RGBD camera, our system provides online 3D object reconstruction capability for users to acquire high-quality textured 3D object models. Starting with a simultaneous pose tracking and TSDF fusion module, our system allows users to scan an object with a mobile device to get a 3D model for real-time preview. After the real-time scanning process is completed, the scanned 3D model is globally optimized and mapped with multi-view textures as an efficient postprocess to get the final textured 3D model on the mobile device. Unlike most existing state-of-the-art systems which can only scan homeware objects such as toys with small dimensions due to the limited computation and memory resources of mobile platforms, our system can reconstruct objects with large dimensions such as statues. We propose a novel visual-inertial ICP approach to achieve real-time accurate 6DoF pose tracking of each incoming frame on the front end, while maintaining a keyframe pool on the back end where the keyframe poses are optimized by local BA. Simultaneously, the keyframe depth maps are fused by the optimized poses to a TSDF model in real-time. Especially, we propose a novel adaptive voxel resizing strategy to solve the out-of-memory problem of large dimension TSDF fusion on mobile platforms. In the post-process, the keyframe poses are globally optimized and the keyframe depth maps are optimized and fused to obtain a final object model with more accurate geometry. The experiments with quantitative and qualitative evaluation demonstrate the effectiveness of the proposed 3D scanning system based on a mobile device, which can successfully achieve online high-quality 3D reconstruction of natural objects with larger dimensions for efficient AR content creation.

Efficient Online Segmentation for Sparse 3 D Laser Scans

Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions

Real-Time Semantic Segmentation of LiDAR Point Clouds on Edge Devices for Unmanned Systems

A Fast and Accurate Segmentation Method for Ordered LiDAR Point Cloud of Large-Scale Scenes

A Fast Method for 3D Object Segmentation Using Sparse Point Clouds

Mobile3DScanner: an Online 3D Scanner for High-quality Object Reconstruction with a Mobile Device

Graph-based segmentation for colored 3D laser point clouds

A Visual-SLAM based Line Laser Scanning System using Semantically Segmented Images

Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data

Scene Understanding in a Large Dynamic Environment Through a Laser-Based Sensing.

3DFusion, A real-time 3D object reconstruction pipeline based on streamed instance segmented data

SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud

Instance Segmentation of Sparse Point Clouds with Spatio-Temporal Coding for Autonomous Robot

Scan-based Semantic Segmentation of LiDAR Point Clouds: An Experimental Study

Point-Cloud Instance Segmentation for Spinning Laser Sensors

Low Latency Instance Segmentation by Continuous Clustering for LiDAR Sensors

A Real-Time 3D Perception and Reconstruction System Based on a 2D Laser Scanner.

LR-Seg: A Ground Segmentation Method for Low-Resolution LiDAR Point Clouds

A Fast Multiplane Segmentation Algorithm for Sparse 3-D LiDAR Point Clouds by Line Segment Grouping.

Fast Ground Segmentation for 3D LiDAR Point Cloud Based on Jump-Convolution-Process

PCSCNet: Fast 3D Semantic Segmentation of LiDAR Point Cloud for Autonomous Car using Point Convolution and Sparse Convolution Network