Hybrid 3D Reconstruction of Indoor Scenes Integrating Object Recognition

Li,Li,Xu,Wei

DOI: https://doi.org/10.3390/rs16040638

IF: 5

2024-02-09

Remote Sensing

Abstract:Indoor 3D reconstruction is particularly challenging due to complex scene structures involving object occlusion and overlap. This paper presents a hybrid indoor reconstruction method that segments the room point cloud into internal and external components, and then reconstructs the room shape and the indoor objects in different ways. We segment the room point cloud into internal and external points based on the assumption that the room shapes are composed of some large external planar structures. For the external, we seek for an appropriate combination of intersecting faces to obtain a lightweight polygonal surface model. For the internal, we define a set of features extracted from the internal points and train a classification model based on random forests to recognize and separate indoor objects. Then, the corresponding computer aided design (CAD) models are placed in the target positions of the indoor objects, converting the reconstruction into a model fitting problem. Finally, the indoor objects and room shapes are combined to generate a complete 3D indoor model. The effectiveness of this method is evaluated on point clouds from different indoor scenes with an average fitting error of about 0.11 m, and the performance is validated by extensive comparisons with state-of-the-art methods.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper aims to address the complex structural issues in 3D reconstruction of indoor scenes, particularly the challenges posed by object occlusion and overlap. Specifically, the paper proposes a hybrid indoor reconstruction method that segments the room point cloud into internal and external components and uses different approaches to reconstruct the room shape and indoor objects, thereby generating a complete 3D indoor model.

### Main Contributions

1. **Divide-and-Conquer Reconstruction Method**:
- A divide-and-conquer reconstruction method based on object-level features is proposed to generate models containing indoor objects and room shapes from point clouds. The room point cloud is segmented into internal and external parts, reconstructed separately, and geometric primitives of intersecting surfaces are used for reconstruction.

2. **Binary Labeling Problem for External Point Cloud Reconstruction**:
- The reconstruction of the external point cloud is treated as a binary labeling problem. By finding suitable combinations of intersecting surfaces, a lightweight and multi-faceted surface model is generated for room shape reconstruction.

3. **Instance Segmentation Assisted Modeling**:
- Instance segmentation technology is used to assist in modeling individual indoor objects. A random forest classifier is designed to identify objects using shape features, spatial features, statistical features, and proprietary features. The reconstruction problem is transformed into a model fitting problem, extracting object-level key points and optimizing to minimize the distance between corresponding key points, thereby accurately placing CAD models.

### Method Overview

1. **Internal and External Segmentation**:
- The input indoor point cloud \( P \) is segmented into two parts: internal point cloud \( P_{\text{in}} \) (containing indoor objects) and external point cloud \( P_{\text{ex}} \) (containing room shapes). By assuming the room shape is composed of piecewise planar subspaces, it can be geometrically represented as a polyhedron.
- An improved RANSAC algorithm with K-nearest neighbors (KNN) is used to extract planes, selecting the plane with the most points as output. Possible erroneous segments are excluded through height thresholds, resulting in the external plane set \( P_{\text{ex}} \) and the remaining internal point cloud \( P_{\text{in}} \).

2. **Room Shape Reconstruction**:
- After extracting some planes from the external point cloud, candidate planes are selected based on the geometric properties of intersecting surfaces. The reconstruction problem of the external point cloud is transformed into a binary labeling problem, and the optimal subset is selected through binary optimization to assemble the room shape model.

3. **Indoor Object Modeling**:
- Instance segmentation technology is used, first training a random forest classifier to segment each point into different object instances, then accurately placing the corresponding CAD models at the target locations, transforming the reconstruction problem into a model fitting problem.

### Conclusion

By combining the advantages of geometric primitives and instance segmentation, this method effectively addresses the complexity and data loss issues in indoor scene reconstruction, generating lightweight and accurate 3D indoor models. Experimental results show that the average fitting error of this method in different indoor scenes is about 0.11 meters, and it performs excellently in extensive comparisons with existing methods.