Abstract:The depth of field of a camera is a limiting factor for applications that require taking images at a short subject-to-camera distance or using a large focal length, such as total body photography, archaeology, and other close-range photogrammetry applications. Furthermore, in multi-view capture, where the target is larger than the camera's field of view, an efficient way to optimize surface coverage captured with quality remains a challenge. Given the 3D mesh of the target object and camera poses, we propose a novel method to derive a focus distance for each camera that optimizes the quality of the covered surface area. We first design an Expectation-Minimization (EM) algorithm to assign points on the mesh uniquely to cameras and then solve for a focus distance for each camera given the associated point set. We further improve the quality surface coverage by proposing a $k$-view algorithm that solves for the points assignment and focus distances by considering multiple views simultaneously. We demonstrate the effectiveness of the proposed method under various simulations for total body photography. The EM and $k$-view algorithms improve the relative cost of the baseline single-view methods by at least $24$% and $28$% respectively, corresponding to increasing the in-focus surface area by roughly $1550$ cm$^2$ and $1780$ cm$^2$. We believe the algorithms can be useful in a number of vision applications that require photogrammetric details but are limited by the depth of field.
What problem does this paper attempt to address?
This paper attempts to solve the problem of surface coverage quality optimization in multi - view capture, especially when the target object is larger than the camera's field of view. Specifically, the paper proposes solutions to the following problems:
1. **Depth - of - Field Limitation**: When shooting at a short distance or using a large focal length, the depth of field (DoF) of the camera becomes a limiting factor, resulting in blurry parts of the image. For example, in full - body photography, archaeology, and other close - range photogrammetry applications, the depth change of the target object may exceed the depth - of - field range of the camera.
2. **Surface Coverage Optimization in Multi - view Capture**: When the target object is larger than the camera's field of view, how to efficiently optimize the focusing distance of each camera to improve the quality of the covered surface is a challenge.
To solve these problems, the paper proposes the following methods:
- **EM Algorithm**: First, an Expectation - Minimization (EM) algorithm is designed to uniquely assign points on the 3D grid to each camera and solve for the optimal focusing distance of each camera. The specific formula is as follows:
\[
\kappa_c^s(p) = w_1 \cdot \min\left(\frac{\varepsilon_1 \langle p - p_c, \vec{n}_c \rangle^2}{\langle \vec{n}_c, \vec{n}_p \rangle}, 1\right) + w_2 \cdot \min\left(\frac{\|\pi_{\vec{n}_c}(p - p_c)\|}{\varepsilon_2}, 1\right) + w_3 \cdot (1 - 1(p \in V_c^s))
\]
where:
- \( w_i \) is the weight of the \( i \) - th term,
- \( \varepsilon_i \) is the threshold,
- \( p_c \) is the position of camera \( c \),
- \( \vec{n}_p \) is the surface normal vector at point \( p \),
- \( \vec{n}_c \) is the line - of - sight direction of camera \( c \),
- \( \pi_{\vec{n}} \) is the function projected onto the plane perpendicular to \( \vec{n} \),
- \( 1(\cdot) \) is the indicator function,
- \( V_c^s \) is the view frustum of camera \( c \) at the focus distance \( s \).
- **k - view Algorithm**: To further improve the surface coverage quality, a k - view algorithm is proposed, which considers multiple views simultaneously to solve point assignment and focusing distance. This algorithm avoids local optimal solutions by jointly optimizing the focusing distances and point assignments of \( k \) cameras.
The paper verifies the effectiveness of the proposed methods through various simulation experiments. The results show that the EM and k - view algorithms respectively improve the relative cost by at least 24% and 28% compared to the single - view baseline method, corresponding to an increase in clear surface area of approximately 1550 cm² and 1780 cm².
In summary, this paper aims to improve the quality of surface coverage by optimizing the focusing distance in multi - view capture, and solves the challenges of depth - of - field limitation and optimization in multi - view capture.