System for 3D Acquisition and 3D Reconstruction using Structured Light for Sewer Line Inspection

Johannes Künzel,Darko Vehar,Rico Nestler,Karl-Heinz Franke,Anna Hilsmann,Peter Eisert

DOI: https://doi.org/10.5220/0011779900003417

2023-03-06

Abstract:The assessment of sewer pipe systems is a highly important, but at the same time cumbersome and error-prone task. We introduce an innovative system based on single-shot structured light modules that facilitates the detection and classification of spatial defects like jutting intrusions, spallings, or misaligned joints. This system creates highly accurate 3D measurements with sub-millimeter resolution of pipe surfaces and fuses them into a holistic 3D model. The benefit of such a holistic 3D model is twofold: on the one hand, it facilitates the accurate manual sewer pipe assessment, on the other, it simplifies the detection of defects in downstream automatic systems as it endows the input with highly accurate depth information. In this work, we provide an extensive overview of the system and give valuable insights into our design choices.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use structured - light technology for high - precision 3D data acquisition and reconstruction in sewer pipe inspection. Specifically, the paper aims to develop an innovative system based on a single - shot structured - light module, which can detect and classify spatial defects, such as protruding intrusions, spalling or misaligned joints. These defects are almost impossible to be detected without depth information, and traditional methods based on monocular images or video sequences have limitations in this regard. Therefore, this research is committed to improving the accuracy of sewer pipe assessment by creating high - precision 3D measurements of the pipe surface with sub - millimeter resolution and fusing these measurements into an overall 3D model. This overall 3D model not only helps in accurate manual assessment of sewer pipes, but also simplifies the detection of defects in subsequent automatic systems because it provides highly accurate depth information.

### Main contributions of the paper:
1. **High - precision 3D data acquisition**: The paper proposes a method that can robustly reconstruct a 3D surface from a single camera image at a moving speed of up to 100 mm/s without additional sensor data.
2. **Robust design**: Through ingenious design decisions, the structured - light sensor can be used in the harsh sewer environment, and at the same time, its favorable characteristics are utilized to build a 3D pipe model with a sufficient level of detail to characterize and quantify damage according to the DIN - EN 13508 standard.
3. **Efficient 3D model generation**: The paper describes in detail the processing chain from the original camera image to the metrically reconstructed 3D point cloud, including steps such as single - shot structured - light method, color - texture pre - processing and 3D mapping, geometric calibration, local and global registration, and uncertainty integration.
4. **Verification in practical applications**: The performance of the system is verified through experiments, and the 3D measurement results of the system on pipes with different diameters are shown, proving the accuracy and reliability of the system.

### Formula representation:
- **Calculation of 3D points by linear triangulation**:
\[
\mathbf{P}=\frac{\mathbf{K}_c^{-1} \mathbf{u}_c\times\mathbf{K}_p^{-1} \mathbf{u}_p}{\|\mathbf{K}_c^{-1} \mathbf{u}_c\times\mathbf{K}_p^{-1} \mathbf{u}_p\|}
\]
where $\mathbf{K}_c$ and $\mathbf{K}_p$ are the internal parameter matrices of the camera and the projector respectively, and $\mathbf{u}_c$ and $\mathbf{u}_p$ are the coordinates of the decoded corresponding points in the camera and projector images.

- **Global non - linear quadratic optimization**:
\[
\delta = \left[R(q_{i,j})^T(p_{k,l}-p_{i,j})-\hat{p}, \, 2\text{vec}((q_{i,j}^{-1}q_{k,l})\hat{q}^{-1})\right]
\]
where $R(q)$ is the rotation matrix of the quaternion $q$, $\text{vec}(q)$ is the vector part of the quaternion, and $\hat{p}$ and $\hat{q}$ are concise representations of relative pose constraints.

### Conclusion:
This paper successfully solves the problem of high - precision 3D data acquisition and reconstruction in sewer pipe inspection by proposing a system based on single - shot structured - light. This system not only improves the accuracy of manual assessment, but also provides high - quality input data for defect detection in subsequent automatic systems.