Korean Cattle 3D Reconstruction from Multi-View 3D-Camera System in Real Environment

Chang Gwon Dang,Seung Soo Lee,Mahboob Alam,Sang Min Lee,Mi Na Park,Ha-Seung Seong,Seungkyu Han,Hoang-Phong Nguyen,Min Ki Baek,Jae Gu Lee,Van Thuan Pham

DOI: https://doi.org/10.3390/s24020427

IF: 3.9

2024-01-11

Sensors

Abstract:The rapid evolution of 3D technology in recent years has brought about significant change in the field of agriculture, including precision livestock management. From 3D geometry information, the weight and characteristics of body parts of Korean cattle can be analyzed to improve cow growth. In this paper, a system of cameras is built to synchronously capture 3D data and then reconstruct a 3D mesh representation. In general, to reconstruct non-rigid objects, a system of cameras is synchronized and calibrated, and then the data of each camera are transformed to global coordinates. However, when reconstructing cattle in a real environment, difficulties including fences and the vibration of cameras can lead to the failure of the process of reconstruction. A new scheme is proposed that automatically removes environmental fences and noise. An optimization method is proposed that interweaves camera pose updates, and the distances between the camera pose and the initial camera position are added as part of the objective function. The difference between the camera's point clouds to the mesh output is reduced from 7.5 mm to 5.5 mm. The experimental results showed that our scheme can automatically generate a high-quality mesh in a real environment. This scheme provides data that can be used for other research on Korean cattle.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

This paper attempts to solve the problems encountered in reconstructing the 3D model of Korean cattle using a multi - view 3D camera system in a real - world environment. Specifically, the authors aim to overcome the following challenges:

1. **Interference from environmental factors**: In a real - world environment, fences may partially block data collection, resulting in large blank areas on the surface of the cattle and generating noisy data near the fences.
2. **Camera vibration**: Due to wind or ground vibration, the position of the camera may shift, making it impossible for the originally calibrated camera to directly map the point cloud to the global coordinate system, thus affecting the reconstruction effect.

To solve these problems, the authors propose a new scheme, which mainly includes the following aspects:

- **Automatically removing environmental noise and fence interference**: Automatically identify and remove the noise introduced by fences and other environmental factors through algorithms to ensure the accuracy of the reconstruction process.
- **Optimizing the camera pose update method**: Propose an interleaved optimization method. When updating the pose of each camera, take the distance between the camera pose and the initial position as part of the objective function to keep the camera pose close to the original position and reduce the error caused by vibration.
- **Improved point - cloud registration and segmentation techniques**: Improve the quality of the final 3D model by segmenting and optimizing the data captured by each camera.

The experimental results show that this scheme can automatically generate high - quality 3D mesh models and perform well in real - world environments. Through these improvements, researchers can more accurately analyze the weight and body characteristics of Korean cattle, and then improve the growth management of cattle.

### Related formulas

During the optimization process, the authors use the following formulas to describe the objective function and the optimization process:

1. **Objective function**:
\[
E(T)=\sum_{i}\sum_{p\in P_{i}}(\text{Dist}(M, p))+\sum_{i}(\Gamma(T_{i}, T_{i}^{0}))
\]
where:
- \(\text{Dist}(M, p)=\min_{T_{rik}\in M}(\text{Dist}(p, T_{rik}))\)
- \(\Gamma(T_{i}, T_{i}^{0})=\Gamma(T_{i}^{- 1}*T_{i}^{0})=\Gamma([R|t])\)

2. **Rotation and translation matrix approximation**:
\[
[R|t]_{i}\approx\begin{pmatrix}
1 - r_{i0}&r_{i1}&t_{i0}\\
-r_{i1}&1 - r_{i2}&t_{i1}\\
r_{i2}&-r_{i2}&1&t_{i2}\\
0&0&0&1
\end{pmatrix}
\]

3. **Distance calculation**:
\[
\Gamma(T_{i}, T_{i}^{0})=k_{r}\cdot(r_{i0}^{2}+r_{i1}^{2}+r_{i2}^{2})+k_{t}\cdot(t_{i0}^{2}+t_{i1}^{2}+t_{i2}^{2})
\]

Through these formulas, the authors ensure that during the optimization process, the camera pose does not deviate too far from the initial position while minimizing the error between point clouds as much as possible.