pping in Multi-view Reconstruction

Texture Ma,Zhaolin Chen,Jun Zhou,Yisong Chen

2012-01-01

Abstract:This paper models. Given a recon high-quality texture mo method. We focus o ghosting artifacts, whi calibration error and blending technique in images in different fre and highlight problem method is employed. E the effectiveness of ou

What problem does this paper attempt to address?

3d Reconstruction With Multi-View Texture Mapping

Xiaodan Ye,Lianghao Wang,Dongxiao Li,Ming Zhang

DOI: https://doi.org/10.1007/978-3-319-70090-8_21

2017-01-01

Abstract:In this paper, a novel 3D reconstruction with multi-view texture mapping method based on Kinect 2 is proposed. Camera poses of all chosen key frames are optimized according to photometric consistency. Optimized camera poses can make the projected point from vertices to different views get closer. A small range of translations with limited calculation is added in this method. A new form of data term and smoothness term in Markov Random Field (MRF) objective function is presented. The outlier images are rejected before view selection and Poisson blending are applied in the end. Experimental results show that our method achieves a high-quality 3D model with high fidelity texture.
3D Texture Mapping in Multi-view Reconstruction

Zhaolin Chen,Jun Zhou,Yisong Chen,Guoping Wang

DOI: https://doi.org/10.1007/978-3-642-33179-4_35

2012-01-01

Abstract:This paper proposes a novel framework for texture mapping of 3D models. Given a reconstructed 3D mesh model and a set of calibrated images, a high-quality texture mosaic of the surface can be created after the process of our method. We focus on avoiding noticeable seams, color inconsistency and ghosting artifacts, which is typically due to such facts as modeling inaccuracy, calibration error and photometric disagreement. We extend the multi-band blending technique in a principled manner and apply it to assemble texture images in different frequency domains elaborately. Meanwhile, self-occlusion and highlight problem is taken into account. Then a novel texture map creating method is employed. Experiments based on our 3D Reconstruction System show the effectiveness of our texturing framework.
Interactive High Resolution Texture Mapping for the 3D Models of Cultural Heritages

Changyu Diao,Dongming Lu

DOI: https://doi.org/10.1007/978-3-540-78566-8_17

2008-01-01

Abstract:Virtual reconstruction of cultural heritage is not only the basis but also one of important contents of its digitized research. The techniques of interactive texture reconstruction researched in this article are of great significance to heighten the effect of virtual reconstruction of cultural heritage. Firstly, interactive and precise texture mapping is aiming at establishing the relationship between texture images and 3D models by appointed corresponding points interactively. The process is based on the traditional camera calibration. For a camera with known intrinsic parameters, three pair of corresponding points will be used to calculate the initial value of extrinsic parameters, and more corresponding points will be used to refine the result. For a camera with unknown intrinsic parameters, an implicit projection matrix will be calculated directly. Secondly, a fast occlusion culling method will be used to remove hidden triangle from texture mapping. At last, a natural neighbor interpolation method will be used to minimize the texture mapping error caused by lens distortion of the camera and the error of 3D scanner.
High-quality texture reconstruction for three-dimensional multi-view imaging (Conference Presentation)

Jiachen Wu,Liangcai Cao,Guofan Jin

DOI: https://doi.org/10.1117/12.2509615

2019-01-01

Abstract:Three-dimensional (3D) model with high-quality texture is a powerful way to interact between virtual and reality 3D scene. It has significant applications in digital preservation of cultural heritage, virtual reality / augmented reality (VR/AR), medical imaging and other domains. High-quality texture reconstruction is one of the essential elements for 3D model. Due to the camera pose error and inaccurate model, current texture reconstruction methods could not eliminate the artifacts like blurring, ghosting and discontinuity. In this work, multi-view high-definition images are captured for texture mapping. Normal-weight, depth-weight and edge-weight parameter are introduced to evaluate texture color confidence, respectively. The normalized weight factors are multiplied to generate a comprehensive weight parameter. By taking a weighted average of projected texture images, discontinuity of texture can be smoothed to a large extent. For large misalignment, bidirectional similarity (BDS) function, which represent the structural similarity between two images, are utilized to improve the texture image. The energy function is composed of two terms. One is Euclidean distance between target texture image and merged texture image. The other is BDS between target texture image and original texture image. By minimizing the energy function, the texture image could generate small local displacement while retaining the original structural information. The target and merged images are optimized alternately, the target image is calculated by patch-match algorithm, and the merged image is derived from weighted average of target images. The method we proposed could produce seamless texture comparing with Markov random field (MRF) algorithm. The definition could be higher than the camera parameter optimization algorithm. The experimental results show that structural similarity (SSIM) between the reconstructed images and the ground truth is higher than traditional algorithms. When dealing with inaccurate models with less than 10,000 facets, the SSIM value could be doubled compared with current algorithms.
Multi-View Stereo Reconstruction With High Dynamic Range Texture

Feng Lu,Xiangyang Ji,Qionghai Dai,Guihua Er

DOI: https://doi.org/10.1007/978-3-642-19309-5_32

2011-01-01

Abstract:In traditional 3D model reconstruction, the texture information is captured in a certain dynamic range, which is usually insufficient for rendering under new environmental light. This paper proposes a novel approach for multi-view stereo (MS) reconstruction of models with high dynamic range (HDR) texture. In the proposed approach, multi-view images are firstly taken with different exposure times simultaneously. Corresponding pixels in adjacent viewpoints are then extracted using a multi-projection method, to robustly recover the response function of the camera. With the response function, pixel values in the differently exposed images can be converted to the desired relative radiance values. Subsequently, geometry reconstruction and HDR texture recovering can be achieved using these values. Experimental results demonstrate that our method can recover the HDR texture for the 3D model efficiently while keep high geometry precision. With our reconstructed HDR texture model, high-quality scene re-lighting is exemplarily exhibited.
Generation of View-Dependent Textures for an Inaccurate Model

Zhen Wang,Weidong Geng

DOI: https://doi.org/10.1109/icis.2017.7959974

2017-01-01

Abstract:Existing methods of texture generation from registered images only work well on accurate models. For inaccurate models, texture drifts may occur. In this paper, we propose a view-dependent seamless texture generation method for inaccurate models. Under a specific viewpoint, this method first assigns each mesh face with a label associated with a registered image to generate a primitive texture. Different from previous methods, our label assignment process is dependent on the current view direction to reduce projective displacements of texture due to imprecision of the geometry. Then, a gradient-domain editing method is used to eliminate seams between image segments. If more than one observation views are given, the seam-levelling method further ensures color consistency between views. Experiments show that our method endows more sense of reality to inaccurate models and succeeds in maintaining temporal color consistency throughout a pre-defined sequence of viewpoints.
On the Relationship Between Multi-View Data Capturing and Quality of Rendered Virtual View

Liu Sx,Ping An,Zhang Zy,Quanwei Zhang,L-Q Shen,Gangyi Jiang

DOI: https://doi.org/10.1179/136821909x12476507838352

2009-01-01

Abstract:According to the Nyquist sampling theorem, a large number of sampled images and small intervals between capturing cameras should be met for rendering high quality virtual views without aliasing, which is hard to realize in practice. Therefore, achieving a balance between multi-view data capturing and quality of the rendered view remains as open problems. To solve this problem, we analysed the spectral bounds of the scene and designed a reconstruction filter. A proper number for rendering and a three-dimensional surface describing the relation between multi-view data capturing and quality of the rendered view were derived. Experimental results for both the modelled scene and the real scene show that only about 20% of sample images are needed compared with Nyquist sampling, while the quality of the rendered view remains higher than that of a Nyquist sampled comparison.
Multiview Textured Mesh Recovery by Differentiable Rendering

Lixiang Lin,Jianke Zhu,Yisu Zhang

DOI: https://doi.org/10.1109/tcsvt.2022.3213543

IF: 5.859

2023-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Although having achieved the promising results on shape and color recovery through self-supervision, the multi-layer perceptrons-based methods usually suffer from heavy computational cost on learning the deep implicit surface representation. Since rendering each pixel requires a forward network inference, it is very computationally intensive to synthesize a whole image. To tackle these challenges, we propose an effective coarse-to-fine approach to recover the textured mesh from multi-views in this paper. Specifically, a differentiable Poisson Solver is employed to represent the object’s shape, which is able to produce topology-agnostic and watertight surfaces. To account for depth information, we optimize the shape geometry by minimizing the differences between the rendered mesh and the predicted depth from multi-view stereo. In contrast to the implicit neural representation on shape and color, we introduce a physically-based inverse rendering scheme to jointly estimate the environment lighting and object’s reflectance, which is able to render the high resolution image at real-time. The texture of reconstructed mesh is interpolated from a learnable dense texture grid. We have conducted the extensive experiments on several multi-view stereo datasets, whose promising results demonstrate the efficacy of our proposed approach. The code is available at https://github.com/l1346792580123/diff .
Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Chenchen Li,Liyang Zhou,Hanqing Jiang,Zhuang Zhang,Xiaojun Xiang,Han Sun,Qing Luan,Hujun Bao,Guofeng Zhang

DOI: https://doi.org/10.1109/tcsvt.2023.3276753

IF: 5.859

2023-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.
Texture Mapping on Real World Models from Multiple Photographic Images

LIU Gang,PENG Qun-Sheng,BAO Hu-Jun

DOI: https://doi.org/10.1360/jos162014

2005-01-01

Abstract:The approach of mapping photographic images on 3D geometries that have been created from real world objects is attracting wide attentions. The registration of the images to the 3D geometries is the key technique. Since previous work used either 3D-2D point matching or silhouette matching for registration, it makes some special demands on the surface features or silhouette shapes of the geometries. In this paper, a new method is presented to overcome this problem. By matching the 3D points reconstructed from the images and the known geometric model in space, the topology and curvature information of the model surface can be fully utilized and all the images to the 3D model stitched at the same time. Experimental results show this method can account for some cases that are intractable for the traditional methods, and the results are quite satisfactory when the distribution of the reconstructed points is relatively symmetrical around the model surface.
Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

Hangqing Jiang,Changfei Zhao,Guofeng Zhang,Huiyan Wang,Hujun Bao

DOI: https://doi.org/10.3969/j.issn.1003-9775.2015.10.001

2015-01-01

Journal of Computer-Aided Design & Computer Graphics

Abstract:Multi-view 3D reconstruction of natural scenes has long been a standard topic in computer vision with various applications. With the prevalence of depth capturing devices, how to effectively use multiple depth maps for 3D scene reconstruction becomes an important problem. This paper proposes a multi-view depth map sampling method for 3D reconstruction of natural scenes to automatically eliminate depth errors in the input depth maps, so that high-quality scene models can be recovered. We first do non-uniform sam-pling on depth maps to obtain a set of 3D points for each frame, and eliminate the 3D points with severe depth errors. Then, a depth confidence estimation algorithm is employed to integrate the sampled 3D points among multiple frames and eliminate redundant ones, so as to obtain the complete 3D point cloud of the scene. Finally, the complete scene geometry model can be generated based on the fused 3D point cloud. A variety of complicated natural scenes are experimented to demonstrate the accuracy and robustness of the proposed method, which can well handle both small objects and large-scale scenes.
Multi-view Reconstruction under Varying Illumination Conditions

Chenglei Wu,Yebin Liu,Xiangyang Ji,Qionghai Dai

DOI: https://doi.org/10.1109/icme.2009.5202648

2009-01-01

Abstract:This paper addresses the problem of complete and detailed 3D model reconstruction of objects filmed by multiple cameras under varying illumination. Firstly, initial normal maps are obtained to enhance the correspondence mapping. Then, the depth for every pixel is estimated by combining photometric constraint with occlusion robust photo-consistency. Finally, after filtering the point cloud, a Poisson surface reconstruction is applied to obtain a watertight mesh. In contrast with traditional photometric stereo techniques, the proposed algorithm does not directly calculate the photometric normal but integrates the photometric constraint into the depth estimation. Furthermore, different from classic multi-view stereo(MVS), we consider the counterpart under changing light conditions. The algorithm has been implemented based on our multicamera and multi-light acquisition system. We validate the method by complete reconstruction of challenging real objects and show experimentally that this technique can greatly improve on correspondence-based MVS results.
Multi-View Depth Estimation with Color-Aware Propagation and Texture-Aware Triangulation

Wenhong Zhang,Song Gu,Yangang Cai,Ronggang Wang

DOI: https://doi.org/10.1109/IPEC54454.2022.9777330

2022-01-01

Abstract:PatchMatch-based multi-view stereo relies on photometric consistency to estimate the depth map for each view. However, due to the matching ambiguity introduced by photo-consistency metrics, it often lacks completeness on textureless areas. In this paper, we propose a novel Multi-View Stereo algorithm for the multi-view depth map estimation problem. First, assuming local areas with similar colors are on the same piecewise plane, depth hypotheses are propagated with color-aware weights during PatchMatch propagation. Second, we propose a texture-aware triangulation to generate reliable planes to approximate the textureless regions, which helps to decrease the matching ambiguities in textureless areas. Moreover, a probabilistic filter is proposed aiming to optimize the estimated depth map with small speckles. The effectiveness of our method has been validated on the ETH3D benchmark. Experiments of our method show competing performance against the state-of-the-art.
Seamless and Aligned Texture Optimization for 3D Reconstruction

Lei Wang,Linlin Ge,Qitong Zhang,Jieqing Feng

DOI: https://doi.org/10.1111/cgf.15205

IF: 2.5

2024-10-26

Computer Graphics Forum

Abstract:Restoring the appearance of the model is a crucial step for achieving realistic 3D reconstruction. High‐fidelity textures can also conceal some geometric defects. Since the estimated camera parameters and reconstructed geometry usually contain errors, subsequent texture mapping often suffers from undesirable visual artifacts such as blurring, ghosting, and visual seams. In particular, significant misalignment between the reconstructed model and the registered images will lead to texturing the mesh with inconsistent image regions. However, eliminating various artifacts to generate high‐quality textures remains a challenge. In this paper, we address this issue by designing a texture optimization method to generate seamless and aligned textures for 3D reconstruction. The main idea is to detect misalignment regions between images and geometry and exclude them from texture mapping. To handle the texture holes caused by these excluded regions, a cross‐patch texture hole‐filling method is proposed, which can also synthesize plausible textures for invisible faces. Moreover, for better stitching of the textures from different views, an improved camera pose optimization is present by introducing color adjustment and boundary point sampling. Experimental results show that the proposed method can eliminate the artifacts caused by inaccurate input data robustly and produce high‐quality texture results compared with state‐of‐the‐art methods.

computer science, software engineering
Unsupervised Texture Reconstruction Method Using Bidirectional Similarity Function for 3-D Measurements

Jiachen Wu,Hailong Chen,Xiaoli Liu,Liangcai Cao,Xiang Peng,Guofan Jin

DOI: https://doi.org/10.1016/j.optcom.2019.01.051

IF: 2.4

2019-01-01

Optics Communications

Abstract:The developments of three-dimensional (3-D) imaging and display technology have created important demands for 3-D model with realistic texture. Due to texture misalignment and inconsistent illumination, texture reconstruction is subject to degradation like blurring, ghosting and color discontinuity. A composite weight texture blending approach is proposed to mitigate the artifacts. For inaccurate geometric model, bidirectional similarity function based multi-scale energy minimization method is proposed to correct texture misalignment. The average processing time of the proposed method is only 40% of existing method. The optical experiments demonstrate that photorealistic texture could be achieved for inaccurate geometric model and inconsistent illumination.
A multi-resolution texture fusion algorithm

Yu Qin Li,Xiang Zhou,Jia Yu Guo,Tao Yang,Zi Xuan Fei

DOI: https://doi.org/10.1117/12.2276853

2017-01-01

Abstract:Texture fusion is essential for three-dimensional photo-realistic texture model reconstruction. Multi-resolution texture fusion mainly is applied to reconstruct three-dimensional models that the local texture is very realistic. This paper presents a multi-resolution texture fusion algorithm based on digital image processing. The technique utilizes the depth camera to obtain range data and the texture camera to obtain the local interested high-resolution texture of the object. And each point of the range images a pixel of the picture corresponding to this view should be found, so the map between high-resolution texture and local three-dimensional points could be obtained by calibrating texture camera and depth camera to register the local high-resolution texture with the low resolution texture collected by the depth camera or calibrating texture camera and depth camera. ICP is applied to register models, and mapping different-resolution textures onto three-dimensional models. Then the light source correction of textures is applied to remove the systematic differences such as lighting change which are achieved by computing linear regression correction mode. Finally, the global correction is applied to refine the observable variations in color that may still exist near the fusion boundaries. These corrections will utilize the grid triangle vertex color as a constraint to drive texture fusion to remove discontinuities from different resolutions. The advantage of this technique is that it utilizes the light source correction and global correction to fuse different resolutions textures. Experiments with this technique indicate that it significantly corrects the observable discontinuities within the overlapping areas, which are given from different resolutions, lighting change, non-lambertian object surface, etc.
Quality Assessment for Virtual View Image Based on Edge Difference

Zhang Yan,An Ping,You Zhi-xiang,Zhang Zhao-yang

DOI: https://doi.org/10.3724/sp.j.1146.2012.01475

2014-01-01

JOURNAL OF ELECTRONICS INFORMATION TECHNOLOGY

Abstract:In order to reduce the transmission data in the multi-view video, the number of multi-views can be decreased properly which include texture images and depth maps, and the intermediate virtual views are generated based on Depth Image Based Rendering (DIBR) at terminal for rendering on 3D displays. Generally, the view synthesis increases the high frequency components on the edge due to inaccurate estimation of the depth map, occlusion and synthesis algorithms. Conventional 2D image quality metric is difficult to reflect the virtual view distortion. In this paper, quality evaluation method based on the edge difference is proposed. Based on the analysis of the pixel difference between virtual and original views, each difference pixel is classified and assigned a visual weight, and higher weights applied to the edge pixels. Experiments for multi-view video sequences prove that the results of this metric are in accordance with characteristic of human visual system.
Texture Mapping of Multi-View High-Resolution Images and Binocular 3D Point Clouds

Du Rui-jian,Ge Bao-zhen,Chen Lei

DOI: https://doi.org/10.37188/co.2020-0034

2020-01-01

Abstract:Aiming at the fusion problem of binocular stereo vision reconstruction point cloud models and high-resolution texture images, a new texture mapping method is proposed. Adding a telephoto texture camera to the binocular stereo vision system to capture high-resolution texture images, the relationship between a texture image and a 3D point cloud model is obtained by matching the two-dimensional features of the high-resolution texture image and the binocular image. The binocular image is used as a bridge, thereby achieving the high-resolution mapping of high-rate texture images on 3D point cloud models. In view of the data redundancy of the overlapping parts of the multi-view texture images during mapping, a method of partitioning the guidance point cloud data is proposed, which effectively solves the problem. Through experimental verification, the proposed method can conveniently and accurately map multi-view texture images to binocular 3D point cloud models. Under experimental conditions, the texture of the 3D model can distinguish line pairs with an original line width of 0.157 mm, which is double the texture resolution of the 3D model directly generated by the binocular system, This verifies the effectiveness of the proposed multi-view high-resolution texture mapping method.
MULTIRESOLUTION PATCH-BASED DENSE RECONSTRUCTION INTEGRATING MULTIVIEW IMAGES AND LASER POINT CLOUD

R. Zhang,X. Yi,H. Li,L. Liu,G. Lu,Y. Chen,X. Guo

DOI: https://doi.org/10.5194/isprs-archives-xliii-b2-2022-153-2022

2022-05-30

Abstract:Abstract. A dense point cloud with rich and realistic texture is generated from multiview images using dense reconstruction algorithms such as Multi View Stereo (MVS). However, its spatial precision depends on the performance of the matching and dense reconstruction algorithms used. Moreover, outliers are usually unavoidable as mismatching of image features. The lidar point cloud lacks texture but performs better spatial precision because it avoids computational errors. This paper proposes a multiresolution patch-based 3D dense reconstruction method based on integrating multiview images and the laser point cloud. A sparse point cloud is firstly generated with multiview images by Structure from Motion (SfM), and then registered with the laser point cloud to establish the mapping relationship between the laser point cloud and multiview images. The laser point cloud is reprojected to multiview images. The corresponding optimal level of the image pyramid is predicted by the distance distribution of projected pixels, which is used as the starting level for patch optimization during dense reconstruction. The laser point cloud is used as stable seed points for patch growth and expansion, and stored by the dynamic octree structure. Subsequently, the corresponding patches are optimized and expanded with the pyramid image to achieve multiscale and multiresolution dense reconstruction. In addition, the octree’s spatial index structure facilitates parallel computing with highly efficiency. The experimental results show that the proposed method is superior to the traditional MVS technology in terms of model accuracy and completeness, and have broad application prospects in high-precision 3D modeling of large scenes.
Planar Texture Rectification and Mapping Based on Reconstruction from Two Views

Xiaogang Sun

2005-01-01

Abstract:When roaming in a 3D virtual scene,texture mapping is a main tactic to display the reality of the scene.But for some special cases i.e.the position of the object or other difficulties which can't be overcome,the texture with perspective distortion is got.Here how to rectify the plain texture to remove perspective distortion and corresponding texture mapping are discussed.This process is as follows: applying projective transformation on the plane,using interpolation method to acquired texture,and then mapping it on the model.This is based on reconstruction from two views.Some existing algorithms are used and some changes are made to achieve a good result during projective transformation.Then the result of test is analyzed and discussed.This method can be used to construct a virtual scene,or clarify the blur of the photos for the reason of perspective distortion.

pping in Multi-view Reconstruction

3d Reconstruction With Multi-View Texture Mapping

3D Texture Mapping in Multi-view Reconstruction

Interactive High Resolution Texture Mapping for the 3D Models of Cultural Heritages

High-quality texture reconstruction for three-dimensional multi-view imaging (Conference Presentation)

Multi-View Stereo Reconstruction With High Dynamic Range Texture

Generation of View-Dependent Textures for an Inaccurate Model

On the Relationship Between Multi-View Data Capturing and Quality of Rendered Virtual View

Multiview Textured Mesh Recovery by Differentiable Rendering

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Texture Mapping on Real World Models from Multiple Photographic Images

Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

Multi-view Reconstruction under Varying Illumination Conditions

Multi-View Depth Estimation with Color-Aware Propagation and Texture-Aware Triangulation

Seamless and Aligned Texture Optimization for 3D Reconstruction

Unsupervised Texture Reconstruction Method Using Bidirectional Similarity Function for 3-D Measurements

A multi-resolution texture fusion algorithm

Quality Assessment for Virtual View Image Based on Edge Difference

Texture Mapping of Multi-View High-Resolution Images and Binocular 3D Point Clouds

MULTIRESOLUTION PATCH-BASED DENSE RECONSTRUCTION INTEGRATING MULTIVIEW IMAGES AND LASER POINT CLOUD

Planar Texture Rectification and Mapping Based on Reconstruction from Two Views