MobileBrick: Building LEGO for 3D Reconstruction on Mobile Devices

Kejie Li,Jia-Wang Bian,Robert Castle,Philip H.S. Torr,Victor Adrian Prisacariu

2023-03-09

Abstract:High-quality 3D ground-truth shapes are critical for 3D object reconstruction evaluation. However, it is difficult to create a replica of an object in reality, and even 3D reconstructions generated by 3D scanners have artefacts that cause biases in evaluation. To address this issue, we introduce a novel multi-view RGBD dataset captured using a mobile device, which includes highly precise 3D ground-truth annotations for 153 object models featuring a diverse set of 3D structures. We obtain precise 3D ground-truth shape without relying on high-end 3D scanners by utilising LEGO models with known geometry as the 3D structures for image capture. The distinct data modality offered by high-resolution RGB images and low-resolution depth maps captured on a mobile device, when combined with precise 3D geometry annotations, presents a unique opportunity for future research on high-fidelity 3D reconstruction. Furthermore, we evaluate a range of 3D reconstruction algorithms on the proposed dataset. Project page: http://code.active.vision/MobileBrick/

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the acquisition of high - quality 3D ground - truth shapes in high - fidelity 3D object reconstruction. Specifically, existing multi - view reconstruction datasets have difficulties in creating accurate 3D models of objects. Even 3D reconstructions generated using high - end 3D scanners have defects due to measurement noise, which can lead to biases in evaluation. Moreover, most existing multi - view reconstruction datasets are not collected on mobile devices, and mobile devices have different data modalities from other devices, such as the combination of high - resolution RGB images and low - resolution depth maps. Therefore, how to effectively use low - quality but easily accessible depth maps on mobile devices for 3D object reconstruction is an under - explored area.

To address these problems, the author proposes a new multi - view RGBD dataset - MobileBrick. This dataset contains 153 object models with diverse 3D structures. By using Lego models with known geometric shapes as 3D structures for image capture, accurate 3D ground - truth shapes can be obtained without relying on high - end 3D scanners. This dataset not only provides accurate 3D geometric annotations but also offers unique opportunities for future research in the field of high - fidelity 3D reconstruction.