Effects of neonatal treatment with Tyr-MIF-1 and naloxone on the long-term body weight gain induced by repeated postnatal stressful stimuli

A. D'amore,A. Loizzo

DOI: https://doi.org/10.1016/S0196-9781(99)00152-7

IF: 3.867

1999-12-01

Peptides

Abstract:

What problem does this paper attempt to address?

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Weifeng Chen,Guofeng Zhang,Xiaojun Xiang,Jiaya Jia,Hujun Bao

DOI: https://doi.org/10.1109/3dv.2014.55

2014-01-01

Abstract:Although multi-view stereo has been extensively studied during the past decades, automatically computing high-quality dense depth information from captured images/videos is still quite difficult. Many factors, such as serious occlusion, large texture less regions and strong reflection, easily cause erroneous depth recovery. In this paper, we present a novel semi-automatic multi-view stereo system, which can quickly create and repair depth from a monocular sequence taken by a freely moving camera. One of our main contributions is that we propose a novel multi-view stereo model incorporating prior constraints indicated by user interaction, which makes it possible to even handle Non-Lambertian surface that surely violates the photo-consistency constraint. Users only need to provide a coarse segmentation and a few user interactions, our system can automatically correct depth and refine boundary. With other priors and occlusion handling, the erroneous depth can be effectively corrected even for very challenging examples that are difficult for state-of-the-art methods.
Multi-View Stereo Representation Revist: Region-Aware MVSNet

Yisu Zhang,Jianke Zhu,Lixiang Lin

DOI: https://doi.org/10.1109/cvpr52729.2023.01667

2023-01-01

Abstract:Deep learning-based multi-view stereo has emerged as a powerful paradigm for reconstructing the complete geometrically-detailed objects from multi-views. Most of the existing approaches only estimate the pixel-wise depth value by minimizing the gap between the predicted point and the intersection of ray and surface, which usually ignore the surface topology. It is essential to the textureless regions and surface boundary that cannot be properly reconstructed. To address this issue, we suggest to take advantage of point-to-surface distance so that the model is able to perceive a wider range of surfaces. To this end, we predict the distance volume from cost volume to estimate the signed distance of points around the surface. Our proposed RA-MVSNet is patch-awared, since the perception range is enhanced by associating hypothetical planes with a patch of surface. Therefore, it could increase the completion of textureless regions and reduce the outliers at the boundary. Moreover, the mesh topologies with fine details can be generated by the introduced distance volume. Comparing to the conventional deep learning-based multi-view stereo methods, our proposed RA-MVSNet approach obtains more complete reconstruction results by taking advantage of signed distance supervision. The experiments on both the DTU and Tanks & Temples datasets demonstrate that our proposed approach achieves the state-of-the-art results.
Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Chenchen Li,Liyang Zhou,Hanqing Jiang,Zhuang Zhang,Xiaojun Xiang,Han Sun,Qing Luan,Hujun Bao,Guofeng Zhang

DOI: https://doi.org/10.1109/tcsvt.2023.3276753

IF: 5.859

2023-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.
Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness

Shuo Cheng,Zexiang Xu,Shilin Zhu,Zhuwen Li,Li Erran Li,Ravi Ramamoorthi,Hao Su

DOI: https://doi.org/10.48550/arXiv.1911.12012

2020-04-19

Abstract:We present Uncertainty-aware Cascaded Stereo Network (UCS-Net) for 3D reconstruction from multiple RGB images. Multi-view stereo (MVS) aims to reconstruct fine-grained scene geometry from multi-view images. Previous learning-based MVS methods estimate per-view depth using plane sweep volumes with a fixed depth hypothesis at each plane; this generally requires densely sampled planes for desired accuracy, and it is very hard to achieve high-resolution depth. In contrast, we propose adaptive thin volumes (ATVs); in an ATV, the depth hypothesis of each plane is spatially varying, which adapts to the uncertainties of previous per-pixel depth predictions. Our UCS-Net has three stages: the first stage processes a small standard plane sweep volume to predict low-resolution depth; two ATVs are then used in the following stages to refine the depth with higher resolution and higher accuracy. Our ATV consists of only a small number of planes; yet, it efficiently partitions local depth ranges within learned small intervals. In particular, we propose to use variance-based uncertainty estimates to adaptively construct ATVs; this differentiable process introduces reasonable and fine-grained spatial partitioning. Our multi-stage framework progressively subdivides the vast scene space with increasing depth resolution and precision, which enables scene reconstruction with high completeness and accuracy in a coarse-to-fine fashion. We demonstrate that our method achieves superior performance compared with state-of-the-art benchmarks on various challenging datasets.

Computer Vision and Pattern Recognition,Machine Learning,Robotics
Depth Extraction Method Based on Multi-view Stereo Matching

鞠芹,安平,张倩,吴妍菲,张兆杨

DOI: https://doi.org/10.3969/j.issn.1000-3428.2010.14.063

2010-01-01

Abstract:A method for generating the depth image from multi-view images based on color segmentation is presented.Mean-shift algorithm is used to extract homogenous regions in the reference image.The initial disparity maps applying local window-based matching algorithm is got.According to certain criteria,these initial depth maps are merged into one disparity map to improve the quality of disparity map,and optimized the disparity map by post processing.The disparity map is converted to a depth map.The algorithm can deal with the occlusion problem effectively,which increases the matching accuracy,and improves the quality of the disparity map.
Digging into Uncertainty in Self-supervised Multi-view Stereo

Hongbin Xu,Zhipeng Zhou,Yali Wang,Wenxiong Kang,Baigui Sun,Hao Li,Yu Qiao

DOI: https://doi.org/10.48550/arXiv.2108.12966

2021-09-08

Abstract:Self-supervised Multi-view stereo (MVS) with a pretext task of image reconstruction has achieved significant progress recently. However, previous methods are built upon intuitions, lacking comprehensive explanations about the effectiveness of the pretext task in self-supervised MVS. To this end, we propose to estimate epistemic uncertainty in self-supervised MVS, accounting for what the model ignores. Specially, the limitations can be categorized into two types: ambiguious supervision in foreground and invalid supervision in background. To address these issues, we propose a novel Uncertainty reduction Multi-view Stereo (UMVS) framework for self-supervised learning. To alleviate ambiguous supervision in foreground, we involve extra correspondence prior with a flow-depth consistency loss. The dense 2D correspondence of optical flows is used to regularize the 3D stereo correspondence in MVS. To handle the invalid supervision in background, we use Monte-Carlo Dropout to acquire the uncertainty map and further filter the unreliable supervision signals on invalid regions. Extensive experiments on DTU and Tank&Temples benchmark show that our U-MVS framework achieves the best performance among unsupervised MVS methods, with competitive performance with its supervised opponents.

Computer Vision and Pattern Recognition
Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

Hangqing Jiang,Changfei Zhao,Guofeng Zhang,Huiyan Wang,Hujun Bao

DOI: https://doi.org/10.3969/j.issn.1003-9775.2015.10.001

2015-01-01

Journal of Computer-Aided Design & Computer Graphics

Abstract:Multi-view 3D reconstruction of natural scenes has long been a standard topic in computer vision with various applications. With the prevalence of depth capturing devices, how to effectively use multiple depth maps for 3D scene reconstruction becomes an important problem. This paper proposes a multi-view depth map sampling method for 3D reconstruction of natural scenes to automatically eliminate depth errors in the input depth maps, so that high-quality scene models can be recovered. We first do non-uniform sam-pling on depth maps to obtain a set of 3D points for each frame, and eliminate the 3D points with severe depth errors. Then, a depth confidence estimation algorithm is employed to integrate the sampled 3D points among multiple frames and eliminate redundant ones, so as to obtain the complete 3D point cloud of the scene. Finally, the complete scene geometry model can be generated based on the fused 3D point cloud. A variety of complicated natural scenes are experimented to demonstrate the accuracy and robustness of the proposed method, which can well handle both small objects and large-scale scenes.
Unsupervised multi-view stereo network based on multi-stage depth estimation

Shuai Qi,Xinzhu Sang,Binbin Yan,Peng Wang,Duo Chen,Huachun Wang,Xiaoqian Ye

DOI: https://doi.org/10.1016/j.imavis.2022.104449

IF: 3.86

2022-06-01

Image and Vision Computing

Abstract:In current years, supervised learning multi-view stereo (MVS) methods have achieved impressive performance. However, these methods still suffer the limitation of hard to acquire large-scale depth supervision data, which hinders the generalization ability in never-seen-before scenarios. Recently, some unsupervised-learning methods have been proposed, which relieved the requirement of depth supervision data. However, the generated depth map with lower resolution since the memory consumption grows cubically. In this paper, we propose a novel unsupervised multi-view stereo network based on multi-stage depth estimation, which can increase depth map resolution and generate a dense 3D model with rich details without relying on depth supervision data. To reduce the 3D cost volume highly memory consumption, the progressive coarse-to-fine multiple stages are adopted. Besides, a multi-view group-wise correlation (MV-GwC) module is designed to introduce multi-view correlation prior, which can enhance network performance and further reduce memory consumption. Qualitative and quantitative experiment results show the effectiveness of our method. We outperform some previous supervised and unsupervised MVS methods on DTU and Tanks & Temples benchmarks.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, software engineering,optics
Vis-MVSNet: Visibility-Aware Multi-view Stereo Network

Jingyang Zhang,Shiwei Li,Zixin Luo,Tian Fang,Yao Yao

DOI: https://doi.org/10.1007/s11263-022-01697-3

IF: 13.369

2022-10-15

International Journal of Computer Vision

Abstract:Learning-based multi-view stereo (MVS) methods have demonstrated promising results. However, very few existing networks explicitly take the pixel-wise visibility into consideration, resulting in erroneous cost aggregation from occluded pixels. In this paper, we explicitly infer and integrate the pixel-wise occlusion information in the MVS network via the matching uncertainty estimation. The pair-wise uncertainty map is jointly inferred with the pair-wise depth map, which is further used as weighting guidance during the multi-view cost volume fusion. As such, the adverse influence of occluded pixels is suppressed in the cost fusion. The proposed framework Vis-MVSNet significantly improves depth accuracy in reconstruction scenes with severe occlusion. Extensive experiments are performed on DTU , BlendedMVS , Tanks and Temples and ETH3D datasets to justify the effectiveness of the proposed framework.

computer science, artificial intelligence
Attention Aware Cost Volume Pyramid Based Multi-view Stereo Network for 3D Reconstruction

Anzhu Yu,Wenyue Guo,Bing Liu,Xin Chen,Xin Wang,Xuefeng Cao,Bingchuan Jiang

DOI: https://doi.org/10.48550/arXiv.2011.12722

2020-11-25

Abstract:We present an efficient multi-view stereo (MVS) network for 3D reconstruction from multiview images. While previous learning based reconstruction approaches performed quite well, most of them estimate depth maps at a fixed resolution using plane sweep volumes with a fixed depth hypothesis at each plane, which requires densely sampled planes for desired accuracy and therefore is difficult to achieve high resolution depth maps. In this paper we introduce a coarseto-fine depth inference strategy to achieve high resolution depth. This strategy estimates the depth map at coarsest level, while the depth maps at finer levels are considered as the upsampled depth map from previous level with pixel-wise depth residual. Thus, we narrow the depth searching range with priori information from previous level and construct new cost volumes from the pixel-wise depth residual to perform depth map refinement. Then the final depth map could be achieved iteratively since all the parameters are shared between different levels. At each level, the self-attention layer is introduced to the feature extraction block for capturing the long range dependencies for depth inference task, and the cost volume is generated using similarity measurement instead of the variance based methods used in previous work. Experiments were conducted on both the DTU benchmark dataset and recently released BlendedMVS dataset. The results demonstrated that our model could outperform most state-of-the-arts (SOTA) methods. The codebase of this project is at <a class="link-external link-https" href="https://github.com/ArthasMil/AACVP-MVSNet" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing
Real-Time Unsupervised Multi-View Depth Estimation Network for Virtual View Synthesis

Ke Qiu,Song Gu,Shiyi Liu,Yawen Lai,Yangang Cai,Ronggang Wang

DOI: https://doi.org/10.1109/icmew53276.2021.9455982

2021-01-01

Abstract:The existing learning-based multi-view stereo (MVS) approaches achieve impressive results compared with traditional methods. However, most of them rely on ground-truth 3D data as supervision, and the acquisition of high-quality ground truth for various scenes is a challenging problem. In this paper, we propose a novel real-time unsupervised multi-view depth estimation network for virtual view synthesis tasks and take multi-view images as supervision. To improve the completeness and accuracy of the virtual viewpoint, we propose a novel shared occlusion mask to deal with the artifacts caused by occlusion in the reconstructed image, and filter out the unreliable points in the depth map. Besides, we also design a mask-based photometric loss to guide our network to generate more reasonable masks and high-quality depth maps. Experimental results on the IEEE1857.9 virtual viewpoint synthesis dataset demonstrate that our proposed method outperforms other recent MVS methods and achieves more excellent real-time performance.
Visibility-Aware Point-Based Multi-View Stereo Network

Rui Chen,Songfang Han,Jing Xu,Hao Su

DOI: https://doi.org/10.1109/tpami.2020.2988729

IF: 23.6

2021-10-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:We introduce VA-Point-MVSNet, a novel visibility-aware point-based deep framework for multi-view stereo (MVS). Distinct from existing cost volume approaches, our method directly processes the target scene as point clouds. More specifically, our method predicts the depth in a coarse-to-fine manner. We first generate a coarse depth map, convert it into a point cloud and refine the point cloud iteratively by estimating the residual between the depth of the current iteration and that of the ground truth. Our network leverages 3D geometry priors and 2D texture information jointly and effectively by fusing them into a feature-augmented point cloud, and processes the point cloud to estimate the 3D flow for each point. This point-based architecture allows higher accuracy, more computational efficiency and more flexibility than cost-volume-based counterparts. Furthermore, our visibility-aware multi-view feature aggregation allows the network to aggregate multi-view appearance cues while taking into account visibility. Experimental results show that our approach achieves a significant improvement in reconstruction quality compared with state-of-the-art methods on the DTU and the Tanks and Temples dataset. The code of VA-Point-MVSNet proposed in this work will be released at https://github.com/callmeray/PointMVSNet.

computer science, artificial intelligence,engineering, electrical & electronic
NR-MVSNet: Learning Multi-View Stereo Based on Normal Consistency and Depth Refinement

Jingliang Li,Zhengda Lu,Yiqun Wang,Jun Xiao,Ying Wang

DOI: https://doi.org/10.1109/tip.2023.3272170

IF: 10.6

2023-05-16

IEEE Transactions on Image Processing

Abstract:Multi-view Stereo (MVS) aims to reconstruct a 3D point cloud model from multiple views. In recent years, learning-based MVS methods have received a lot of attention and achieved excellent performance compared with traditional methods. However, these methods still have apparent shortcomings, such as the accumulative error in the coarse-to-fine strategy and the inaccurate depth hypotheses based on the uniform sampling strategy. In this paper, we propose the NR-MVSNet, a coarse-to-fine structure with the depth hypotheses based on the normal consistency (DHNC) module, and the depth refinement with reliable attention (DRRA) module. Specifically, we design the DHNC module to generate more effective depth hypotheses, which collects the depth hypotheses from neighboring pixels with the same normals. As a result, the predicted depth can be smoother and more accurate, especially in texture-less and repetitive-texture regions. On the other hand, we update the initial depth map in the coarse stage by the DRRA module, which can combine attentional reference features and cost volume features to improve the depth estimation accuracy in the coarse stage and address the accumulative error problem. Finally, we conduct a series of experiments on the DTU, BlendedMVS, Tanks & Temples, and ETH3D datasets. The experimental results demonstrate the efficiency and robustness of our NR-MVSNet compared with the state-of-the-art methods. Our implementation is available at https://github.com/wdkyh/NR-MVSNet.

computer science, artificial intelligence,engineering, electrical & electronic
A Confidence-based Iterative Solver of Depths and Surface Normals for Deep Multi-view Stereo

Wang Zhao,Shaohui Liu,Yi Wei,Hengkai Guo,Yong-Jin Liu

DOI: https://doi.org/10.1109/ICCV48922.2021.00611

2021-01-01

Abstract:In this paper, we introduce a deep multi-view stereo (MVS) system that jointly predicts depths, surface normals and per-view confidence maps. The key to our approach is a novel solver that iteratively solves for per-view depth map and normal map by optimizing an energy potential based on the locally planar assumption. Specifically, the algorithm updates depth map by propagating from neighboring pixels with slanted planes, and updates normal map with local probabilistic plane fitting. Both two steps are monitored by a customized confidence map. This solver is not only effective as a post-processing tool for plane-based depth refinement and completion, but also differentiable such that it can be efficiently integrated into deep learning pipelines. Our multi-view stereo system employs multiple optimization steps of the solver over the initial prediction of depths and surface normals. The whole system can be trained end-to-end, decoupling the challenging problem of matching pixels within poorly textured regions from the cost-volume based neural network. Experimental results on ScanNet and RGB-D Scenes V2 demonstrate state-of-the-art performance of the proposed deep MVS system on multi-view depth estimation, with our proposed solver consistently improving the depth quality over both conventional and deep learning based MVS pipelines.
Multi-View Depth Estimation with Color-Aware Propagation and Texture-Aware Triangulation

Wenhong Zhang,Song Gu,Yangang Cai,Ronggang Wang

DOI: https://doi.org/10.1109/IPEC54454.2022.9777330

2022-01-01

Abstract:PatchMatch-based multi-view stereo relies on photometric consistency to estimate the depth map for each view. However, due to the matching ambiguity introduced by photo-consistency metrics, it often lacks completeness on textureless areas. In this paper, we propose a novel Multi-View Stereo algorithm for the multi-view depth map estimation problem. First, assuming local areas with similar colors are on the same piecewise plane, depth hypotheses are propagated with color-aware weights during PatchMatch propagation. Second, we propose a texture-aware triangulation to generate reliable planes to approximate the textureless regions, which helps to decrease the matching ambiguities in textureless areas. Moreover, a probabilistic filter is proposed aiming to optimize the estimated depth map with small speckles. The effectiveness of our method has been validated on the ETH3D benchmark. Experiments of our method show competing performance against the state-of-the-art.
Learning-based Multi-View Stereo: A Survey

Fangjinhua Wang,Qingtian Zhu,Di Chang,Quankai Gao,Junlin Han,Tong Zhang,Richard Hartley,Marc Pollefeys

2024-08-28

Abstract:3D reconstruction aims to recover the dense 3D structure of a scene. It plays an essential role in various applications such as Augmented/Virtual Reality (AR/VR), autonomous driving and robotics. Leveraging multiple views of a scene captured from different viewpoints, Multi-View Stereo (MVS) algorithms synthesize a comprehensive 3D representation, enabling precise reconstruction in complex environments. Due to its efficiency and effectiveness, MVS has become a pivotal method for image-based 3D reconstruction. Recently, with the success of deep learning, many learning-based MVS methods have been proposed, achieving impressive performance against traditional methods. We categorize these learning-based methods as: depth map-based, voxel-based, NeRF-based, 3D Gaussian Splatting-based, and large feed-forward methods. Among these, we focus significantly on depth map-based methods, which are the main family of MVS due to their conciseness, flexibility and scalability. In this survey, we provide a comprehensive review of the literature at the time of this writing. We investigate these learning-based methods, summarize their performances on popular benchmarks, and discuss promising future research directions in this area.

Computer Vision and Pattern Recognition
HC-MVSNet: A Probability Sampling-Based Multi-View-stereo Network with Hybrid Cascade Structure for 3D Reconstruction

Tianxiang Gao,Zijian Hong,Yixing Tan,Lizhuo Sun,Yichen Wei,Jianwei Ma

DOI: https://doi.org/10.1016/j.patrec.2024.07.008

IF: 4.757

2024-01-01

Pattern Recognition Letters

Abstract:Multi-view stereo (MVS) is one of the ways to obtain the 3D structure from 2D images. Deep learning is an effective end-to-end method for MVS. In previous MVS methods based on deep learning, the depth interval is deeply coupled with the feature map resolution, resulting in more accurate depth intervals accompanied by higher computational cost. This paper proposes a new deep neural network HC-MVSNet which utilizes a hybrid cascade structures for depth estimation of MVS. Different from the previous MVS methods, the new coarse-to-fine depth estimation method decouples the two processes of resolution increase and depth interval reduction through a simple operation, achieving higher reconstruction accuracy and completeness for minimal additional computational cost. In addition, an efficient depth sampling strategy based on probability distribution is introduced, which allocates higher hypothesis density for regions with a high probability of ground truth. This novel sampling method makes full use of redundant information that was previously neglected and significantly improves the textural detail of the results. Extensive experiments are conducted on DTU datasets, Tanks and Temples benchmark, and BlendedMVS datasets. The results show that the proposed method exhibits superior performance and better generalization behavior than existing MVS methods.
Adaptive Cost Aggregation in Iterative Depth Estimation for Efficient Multi-view Stereo.

Xiang Wang,Xiao Bai ,Chen Wang

DOI: https://doi.org/10.1007/978-3-031-46308-2_3

2023-01-01

Abstract:The deep multi-view stereo (MVS) approaches generally construct 3D cost volumes to regularize and regress the depth map. These methods are limited with high-resolution outputs since the memory and time costs grow cubically as the volume resolution increases. In this paper, we presented an multi-stage iterative depth map estimation method for MVS. In our network, the cost volume is iteratively processed by lightweight 2D convolution based GRU modules, and the multi-stage coarse-to-fine structure is adopted to speed up the depth estimation process. To further improve the 3D reconstruction quality, we make improvements from two different perspectives of adaptive cost aggregation: a view-adaptive weighting module is proposed to account for the occlusion problem in cost volume fusion, and a spatial-adaptive deformable geometric feature encoding module is introduced to the cost volume feature encoding before feeding into GRUs for stronger modeling capability. Experiments on the DTU dataset demonstrated the effectiveness of the proposed network in accuracy with remarkable efficiency performance.
RIAV-MVS: Recurrent-Indexing an Asymmetric Volume for Multi-View Stereo

Changjiang Cai,Pan Ji,Qingan Yan,Yi Xu

2024-11-05

Abstract:This paper presents a learning-based method for multi-view depth estimation from posed images. Our core idea is a "learning-to-optimize" paradigm that iteratively indexes a plane-sweeping cost volume and regresses the depth map via a convolutional Gated Recurrent Unit (GRU). Since the cost volume plays a paramount role in encoding the multi-view geometry, we aim to improve its construction both at pixel- and frame- levels. At the pixel level, we propose to break the symmetry of the Siamese network (which is typically used in MVS to extract image features) by introducing a transformer block to the reference image (but not to the source images). Such an asymmetric volume allows the network to extract global features from the reference image to predict its depth map. Given potential inaccuracies in the poses between reference and source images, we propose to incorporate a residual pose network to correct the relative poses. This essentially rectifies the cost volume at the frame level. We conduct extensive experiments on real-world MVS datasets and show that our method achieves state-of-the-art performance in terms of both within-dataset evaluation and cross-dataset generalization. Code available: <a class="link-external link-https" href="https://github.com/oppo-us-research/riav-mvs" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
Multi-View Stereo Representation Revisit: Region-Aware MVSNet

Yisu Zhang,Jianke Zhu,Lixiang Lin

2023-04-27

Abstract:Deep learning-based multi-view stereo has emerged as a powerful paradigm for reconstructing the complete geometrically-detailed objects from multi-views. Most of the existing approaches only estimate the pixel-wise depth value by minimizing the gap between the predicted point and the intersection of ray and surface, which usually ignore the surface topology. It is essential to the textureless regions and surface boundary that cannot be properly reconstructed. To address this issue, we suggest to take advantage of point-to-surface distance so that the model is able to perceive a wider range of surfaces. To this end, we predict the distance volume from cost volume to estimate the signed distance of points around the surface. Our proposed RA-MVSNet is patch-awared, since the perception range is enhanced by associating hypothetical planes with a patch of surface. Therefore, it could increase the completion of textureless regions and reduce the outliers at the boundary. Moreover, the mesh topologies with fine details can be generated by the introduced distance volume. Comparing to the conventional deep learning-based multi-view stereo methods, our proposed RA-MVSNet approach obtains more complete reconstruction results by taking advantage of signed distance supervision. The experiments on both the DTU and Tanks \& Temples datasets demonstrate that our proposed approach achieves the state-of-the-art results.

Computer Vision and Pattern Recognition

Effects of neonatal treatment with Tyr-MIF-1 and naloxone on the long-term body weight gain induced by repeated postnatal stressful stimuli

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Multi-View Stereo Representation Revist: Region-Aware MVSNet

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness

Depth Extraction Method Based on Multi-view Stereo Matching

Digging into Uncertainty in Self-supervised Multi-view Stereo

Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

Unsupervised multi-view stereo network based on multi-stage depth estimation

Vis-MVSNet: Visibility-Aware Multi-view Stereo Network

Attention Aware Cost Volume Pyramid Based Multi-view Stereo Network for 3D Reconstruction

Real-Time Unsupervised Multi-View Depth Estimation Network for Virtual View Synthesis

Visibility-Aware Point-Based Multi-View Stereo Network

NR-MVSNet: Learning Multi-View Stereo Based on Normal Consistency and Depth Refinement

A Confidence-based Iterative Solver of Depths and Surface Normals for Deep Multi-view Stereo

Multi-View Depth Estimation with Color-Aware Propagation and Texture-Aware Triangulation

Learning-based Multi-View Stereo: A Survey

HC-MVSNet: A Probability Sampling-Based Multi-View-stereo Network with Hybrid Cascade Structure for 3D Reconstruction

Adaptive Cost Aggregation in Iterative Depth Estimation for Efficient Multi-view Stereo.

RIAV-MVS: Recurrent-Indexing an Asymmetric Volume for Multi-View Stereo

Multi-View Stereo Representation Revisit: Region-Aware MVSNet