Abstract:Large volumes of 3D parametric datasets, such as building information modeling (BIM), are the foundation for developing and applying smart city and digital twin technologies. Those datasets are also considered valuable tools for efficiently managing rebuilt structures during the operation and maintenance stages. Nevertheless, current approaches developed for the scan-to-BIM process rely on manual or semi-automatic procedures and insufficiently leverage semantic data in point clouds. These methods struggle to accurately represent large-scale indoor complex layouts and extract details from irregular-shaped unstructured elements, causing inefficiencies in BIM model generation. To address these issues, we propose an innovative scan-to-BIM framework based on deep learning algorithms and raw point cloud data, enabling the automatic generation of 3D models for both structured and unstructured indoor elements. Initially, we propose an enhanced deep learning neural network to improve the point clouds' semantic segmentation accuracy. Subsequently, an efficient workflow is developed to reconstruct 3D building models of structured indoor scenes. The proposed workflow can reconstruct large-scale data with multiple room layouts of Manhattan or non-Manhattan structures and reconstruct 3D models automatically by using a BIM parametric algorithm implemented in Revit software. Moreover, we introduce a robust method for unstructured elements to automatically generate corresponding 3D BIM models, even when the incorporating semantic information is incomplete. The proposed approach was evaluated on synthetic and real data for different scales and complexities of indoor scenes. The results of the experiments demonstrate that the improved model significantly enhances the overall semantic segmentation accuracy compared to the baseline models. The proposed scan-to-BIM framework is efficient for indoor element 3D reconstruction, achieving precision, recall, and F-score values ranging from 96% to 99%. The generated BIM models are competitive with traditional methods regarding model completeness and geometric accuracy.

Indoor camera pose estimation via style‐transfer 3D models

A Flexible Method for 3D Reconstruction of Urban Building

Two-Stage Multi-Camera Constrain Mapping Pipeline for Large-Scale 3D Reconstruction

Improving synthetic 3D model-aided indoor image localization via domain adaptation

3D Model Retrieval and Pose Estimation for Indoor Images by Simulating Scene Context

Visual Odometry Based 3D-Reconstruction

3D Reconstruction of Indoor Scenes Via Image Registration

Object-aware Semantic Mapping of Indoor Scenes Using Octomap

Computer Vision Enabled Building Digital Twin Using Building Information Model

Unsupervised Domain Adaptation for 3D Human Pose Estimation

Pose Estimation Based on Bidirectional Visual–Inertial Odometry with 3D LiDAR (BV-LIO)

Indoor Scene Reconstruction Using a Rotating Device and Multiple RGB-D Cameras

Indoor Layout Estimation by 2D LiDAR and Camera Fusion

LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-frame 3D Human Pose Estimation

An Interactive Approach to Semantic Modeling of Indoor Scenes with an RGBD Camera

Indoor Scene Reconstruction From Monocular Video Combining Contextual and Geometric Priors

VirtualPose: Learning Generalizable 3D Human Pose Models from Virtual Data

ReLoc: Indoor Visual Localization with Hierarchical Sitemap and View Synthesis

Automated BIM generation for large-scale indoor complex environments based on deep learning

Language-guided Semantic Style Transfer of 3D Indoor Scenes

Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation