Phase II study of 1,2,4-triglycidylurazol (TGU) in previously untreated and treated patients with small cell lung cancer.

B. Lund,F. Hansen,M. Hansen,H. Hansen

DOI: https://doi.org/10.1016/0277-5379(87)90354-3

1987-07-01

European Journal of Cancer and Clinical Oncology

Abstract:

What problem does this paper attempt to address?

A clustering approach to free form surface reconstruction from multi-view range images

Hong Zhou,Yonghuai Liu,Longzhuang Li,Baogang Wei

DOI: https://doi.org/10.1016/j.imavis.2008.07.009

IF: 3.86

2009-01-01

Image and Vision Computing

Abstract:3D modelling finds a wide range of applications in robot vision and reverse engineering. However due to the presence of surface scanning noise, accumulative registration errors, and improper data fusion, the reconstructed surfaces from multiple registered range images are often non-smooth and distorted with thick patches, false connections and blurred features. These shortcomings will limit the wide applications of 3D modelling using the latest laser scanning systems. In this paper, the clustering approach, surface segmentation and classification are employed to fuse registered range images to reduce the adverse effect of large accumulative registration errors and heavy scanning noise, minimize the dissimilarity of the fused surface with respect to the original overlapping surfaces, and produce smooth and detailed 3D object computer models. For initialization of the clustering approach, an automatic method is developed, shifting possible corresponding points from different viewpoints toward each other and thus, making sure that the initialization of the cluster centroids are in between the two data sets so that more efficient and effective integration of data can be obtained. Then the principal component analysis (PCA) is employed to segment these centroids into different areas for the representation of the initially fused surface and then the tensor analysis is applied to classify these areas into featured and non-featured ones. In the integration process, the K means and fuzzy c means clustering approaches from the pattern recognition and machine learning literatures are employed to integrate non-featured areas for a smooth fused surface and featured areas for keeping the geometric details. By controlling a parameter, the final integrated surface can be traded off between smoothness and geometric details. Finally the fused point set is triangulated using an improved Delaunay method, guaranteeing a watertight surface. The new method is theoretically guaranteed to converge and minimize the dissimilarity between the final fused surface and original surfaces. A comparative study based on real images shows that the proposed algorithm desirably retains geometric details, produces smooth surface and minimizes the integration error.
High-fidelity 3D Face Reconstruction with Multi-Scale Details

Yiwei Jin,Qingyu Li,Diqiong Jiang,Ruofeng Tong

DOI: https://doi.org/10.1016/j.patrec.2021.11.022

IF: 4.757

2022-01-01

Pattern Recognition Letters

Abstract:Despite tremendous success has been achieved in faithfully reconstructing face shapes from single images, recovering accurate local details still remains challenging. Previous works propose reprojectionbased methods to improve the performance of detail recovering - they render a textured 3D shape into an image and make it approximate to the input during iterations. However, details from textures and shapes are mixed in the rendered image when minimizing the re-projection loss, which leads to limitations in detail recovery. To address this issue, we propose a novel 3D face reconstruction framework that 1) uses a coarse-medium-fine strategy to capture details while preserving the global shape, 2) disentangles details from the texture to enhance local accuracy, and 3) applies a phased optimization to recover details over multiple scales. Experiments demonstrate the capability of our framework to reconstruct high-fidelity face shapes with accurate, fine details. (c) 2021 Published by Elsevier B.V.
Prior surface model guided dense reconstruction via graph-cuts

Wenhui Zuo,Jingting Ding,Jilin Liu

DOI: https://doi.org/10.1109/WCICA.2014.7053271

2014-01-01

Intelligent Control and Automation

Abstract:A fronto-parallel surface model is implied by the conventional smoothing constraints used in most stereo vision algorithms. It results in systematic errors in natural scene reconstruction. In this paper, we propose a novel method incorporating a non- planar surface obtained through sparse stereo reconstruction as the prior model. The depth map is recovered by solving a multi-label Markov Random Filed (MRF) optimization via graph-cuts. Our energy minimization formulation exploits prior information by incorporating surface expectation in la- bel definition. It results in a depth map with piecewise con- tinuity along the prior surface while maintains efficiency of original algorithm. Moreover, with the aid of the prior surface model, our method can directly generate an approximate 3D patch in depth refinement which is a more accurate description of the correlation region. We demonstrate our algorithm on real scene and show the promising experimental results.
High-Quality Textured 3D Shape Reconstruction with Cascaded Fully Convolutional Networks

Zheng-Ning Liu,Yan-Pei Cao,Zheng-Fei Kuang,Leif Kobbelt,Shi-Min Hu

DOI: https://doi.org/10.1109/tvcg.2019.2937300

IF: 5.2

2021-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:We present a learning-based approach to reconstructing high-resolution three-dimensional (3D) shapes with detailed geometry and high-fidelity textures. Albeit extensively studied, algorithms for 3D reconstruction from multi-view depth-and-color (RGB-D) scans are still prone to measurement noise and occlusions; limited scanning or capturing angles also often lead to incomplete reconstructions. Propelled by recent advances in 3D deep learning techniques, in this paper, we introduce a novel computation- and memory-efficient cascaded 3D convolutional network architecture, which learns to reconstruct implicit surface representations as well as the corresponding color information from noisy and imperfect RGB-D maps. The proposed 3D neural network performs reconstruction in a progressive and coarse-to-fine manner, achieving unprecedented output resolution and fidelity. Meanwhile, an algorithm for end-to-end training of the proposed cascaded structure is developed. We further introduce Human10, a newly created dataset containing both detailed and textured full-body reconstructions as well as corresponding raw RGB-D scans of 10 subjects. Qualitative and quantitative experimental results on both synthetic and real-world datasets demonstrate that the presented approach outperforms existing state-of-the-art work regarding visual quality and accuracy of reconstructed models.
Enhanced 3D Shape Reconstruction With Knowledge Graph of Category Concept

Guofei Sun,Yongkang Wong,Mohan S. Kankanhalli,Xiangdong Li,Weidong Geng

DOI: https://doi.org/10.1145/3491224

IF: 4.094

2022-01-01

ACM Transactions on Multimedia Computing Communications and Applications

Abstract:Reconstructing three-dimensional (3D) objects from images has attracted increasing attention due to its wide applications in computer vision and robotic tasks. Despite the promising progress of recent deep learning-based approaches, which directly reconstruct the full 3D shape without considering the conceptual knowledge of the object categories, existing models have limited usage and usually create unrealistic shapes. 3D objects have multiple forms of representation, such as 3D volume, conceptual knowledge, and so on. In this work, we show that the conceptual knowledge for a category of objects, which represents objects as prototype volumes and is structured by graph, can enhance the 3D reconstruction pipeline. We propose a novel multimodal framework that explicitly combines graph-based conceptual knowledge with deep neural networks for 3D shape reconstruction from a single RGB image. Our approach represents conceptual knowledge of a specific category as a structure-based knowledge graph. Specifically, conceptual knowledge acts as visual priors and spatial relationships to assist the 3D reconstruction framework to create realistic 3D shapes with enhanced details. Our 3D reconstruction framework takes an image as input. It first predicts the conceptual knowledge of the object in the image, then generates a 3D object based on the input image and the predicted conceptual knowledge. The generated 3D object satisfies the following requirements: (1) it is consistent with the predicted graph in concept, and (2) consistent with the input image in geometry. Extensive experiments on public datasets (i.e., ShapeNet, Pix3D, and Pascal3D+) with 13 object categories show that (1) our method outperforms the state-of-the-art methods, (2) our prototype volume-based conceptual knowledge representation is more effective, and (3) our pipeline-agnostic approach can enhance the reconstruction quality of various 3D shape reconstruction pipelines.
Geometry-aware Two-scale PIFu Representation for Human Reconstruction

Zheng Dong,Ke Xu,Ziheng Duan,Hujun Bao,Weiwei Xu,Rynson W. H. Lau

DOI: https://doi.org/10.48550/arxiv.2112.02082

2022-01-01

Abstract: Although PIFu-based 3D human reconstruction methods are popular, the quality of recovered details is still unsatisfactory. In a sparse (e.g., 3 RGBD sensors) capture setting, the depth noise is typically amplified in the PIFu representation, resulting in flat facial surfaces and geometry-fallible bodies. In this paper, we propose a novel geometry-aware two-scale PIFu for 3D human reconstruction from sparse, noisy inputs. Our key idea is to exploit the complementary properties of depth denoising and 3D reconstruction, for learning a two-scale PIFu representation to reconstruct high-frequency facial details and consistent bodies separately. To this end, we first formulate depth denoising and 3D reconstruction as a multi-task learning problem. The depth denoising process enriches the local geometry information of the reconstruction features, while the reconstruction process enhances depth denoising with global topology information. We then propose to learn the two-scale PIFu representation using two MLPs based on the denoised depth and geometry-aware features. Extensive experiments demonstrate the effectiveness of our approach in reconstructing facial details and bodies of different poses and its superiority over state-of-the-art methods.
Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture

Yixin Chen,Junfeng Ni,Nan Jiang,Yaowei Zhang,Yixin Zhu,Siyuan Huang

2023-11-01

Abstract:Reconstructing detailed 3D scenes from single-view images remains a challenging task due to limitations in existing approaches, which primarily focus on geometric shape recovery, overlooking object appearances and fine shape details. To address these challenges, we propose a novel framework for simultaneous high-fidelity recovery of object shapes and textures from single-view images. Our approach utilizes the proposed Single-view neural implicit Shape and Radiance field (SSR) representations to leverage both explicit 3D shape supervision and volume rendering of color, depth, and surface normal images. To overcome shape-appearance ambiguity under partial observations, we introduce a two-stage learning curriculum incorporating both 3D and 2D supervisions. A distinctive feature of our framework is its ability to generate fine-grained textured meshes while seamlessly integrating rendering capabilities into the single-view 3D reconstruction model. This integration enables not only improved textured 3D object reconstruction by 27.7% and 11.6% on the 3D-FRONT and Pix3D datasets, respectively, but also supports the rendering of images from novel viewpoints. Beyond individual objects, our approach facilitates composing object-level representations into flexible scene representations, thereby enabling applications such as holistic scene understanding and 3D scene editing. We conduct extensive experiments to demonstrate the effectiveness of our method.

Computer Vision and Pattern Recognition
Surface Reconstruction from Sparse & Arbitrarily Oriented Contours in Freehand 3D Ultrasound.

Shuangcheng Deng,Yunhua Li,Lipei Jiang,Yingyu Cao,Junwen Zhang

DOI: https://doi.org/10.4304/jcp.8.5.127-1285

2013-01-01

Journal of Computers

Abstract:3D reconstruction for freehand 3D ultrasound is a challenging issue because the recorded B-scans are not only sparse, but also non-parallel (actually they may arbitrarily oriented in 3D space and may intersect each other). Conventional volume reconstruction methods can't reconstruct sparse data efficiently while not introducing geometrical artifacts, and conventional surface reconstruction methods can't reconstruct surfaces from contours that are arbitrarily oriented in 3D space. We developed a new surface reconstruction method for freehand 3D ultrasound based on variational implicit function which is presented by Greg Turk for shape transformation. In the new method, we first constructed on& off-surface constraints from the segmented contours of all recorded B-scans, then used a variational interpolation technique to get a single implicit function in 3D. Finally, the implicit function was evaluated to extract the zero-valued surface as final reconstruction result. Two experiment was conducted to assess our variational surface reconstruction method, and the experiment results have shown that the new method is capable of reconstructing surface smoothly from sparse contours which can be arbitrarily oriented in 3D space.
TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Ying Huang,Lin Fang,Shanfeng Hu

DOI: https://doi.org/10.3390/s23146525

IF: 3.9

2023-07-20

Sensors

Abstract:We present TED-Face, a new method for recovering high-fidelity 3D facial geometry and appearance with enhanced textures from single-view images. While vision-based face reconstruction has received intensive research in the past decades due to its broad applications, it remains a challenging problem because human eyes are particularly sensitive to numerically minute yet perceptually significant details. Previous methods that seek to minimize reconstruction errors within a low-dimensional face space can suffer from this issue and generate close yet low-fidelity approximations. The loss of high-frequency texture details is a key factor in their process, which we propose to address by learning to recover both dense radiance residuals and sparse facial texture features from a single image, in addition to the variables solved by previous work—shape, appearance, illumination, and camera. We integrate the estimation of all these factors in a single unified deep neural network and train it on several popular face reconstruction datasets. We also introduce two new metrics, visual fidelity (VIF) and structural similarity (SSIM), to compensate for the fact that reconstruction error is not a consistent perceptual metric of quality. On the popular FaceWarehouse facial reconstruction benchmark, our proposed system achieves a VIF score of 0.4802 and an SSIM score of 0.9622, improving over the state-of-the-art Deep3D method by 6.69% and 0.86%, respectively. On the widely used LS3D-300W dataset, we obtain a VIF score of 0.3922 and an SSIM score of 0.9079 for indoor images, and the scores for outdoor images are 0.4100 and 0.9160, respectively, which also represent an improvement over those of Deep3D. These results show that our method is able to recover visually more realistic facial appearance details compared with previous methods.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Variational Approach To Reconstruct Surface From Sparse & Nonparallel Contours In Freehand 3d Ultrasound Imaging

Shuangcheng Deng,Lipei Jiang,Yingyu Cao,Junwen Zhang,Haiyang Zheng

DOI: https://doi.org/10.1117/12.917670

2012-01-01

Abstract:The 3D reconstruction for freehand 3D ultrasound is a challenging issue because the recorded B-scans are not only sparse, but also non-parallel (actually they may intersect each other). Conventional volume reconstruction methods can't reconstruct sparse data efficiently while not introducing geometrical artifacts, and conventional surface reconstruction methods can't reconstruct surfaces from contours that are arbitrarily oriented in 3D space. We developed a new surface reconstruction method for freehand 3D ultrasound. It is based on variational implicit function which is presented by Greg Turk for shape transformation. In the new method, we first constructed on- & off-surface constraints from the segmented contours of all recorded B-scans, then used a variational interpolation technique to get a single implicit function in 3D. Finally, the implicit function was evaluated to extract the zero-valued surface as reconstruction result. Two experiment was conducted to assess our variational surface reconstruction method, and the experiment results have shown that the new method is capable of reconstructing surface smoothly from sparse contours which can be arbitrarily oriented in 3D space.
Toward a Clinically‐Oriented Model of Anxiety Disorders

P. Norton

DOI: https://doi.org/10.1080/16506070500441561

2006-06-01

Cognitive Behaviour Therapy

Abstract:The Diagnostic and Statistical Manual of Mental Disorders (DSM) model of psychopathology has generally been the accepted standard in North America for understanding and diagnosing psychological disorders for over half a century. This classification model, particularly since DSM‐III, has been formulated around the goals of aiding diagnosis, enhancing communication among professionals, fostering psychopathology research and informing treatment. However, all classification systems are inherently dependent on the purpose for the classification. In this paper, an argument is made for a clinically‐relevant diagnostic system of mental disorders to support a primary goal of informing treatment. Several lines of research are examined, including studies on diagnostic reliability, dimensional vs categorical nature of anxiety disorders, co‐morbidity, and psychotherapeutic and pharmacological treatment outcome as they relate to current and proposed diagnostic models of anxiety disorders. Based on the evidence, suggestions are made for revising diagnostic models of anxiety, and key lines of future research are proposed.
Surface Reconstruction from Tracked Endoscopic Video Using the Structure from Motion Approach.

Deyu Sun,Jiquan Liu,Cristian A. Linte,Huilong Duan,Richard A. Robb

DOI: https://doi.org/10.1007/978-3-642-40843-4_14

2013-01-01

Abstract:The lack of 3D vision and proper depth perception associated with traditional endoscopy significantly limits the quality of the diagnostic examinations and therapy delivery. To address this challenge, we propose a technique to reconstruct a 3D model of the visualized scene from a sequence of spatially-encoded endoscopic video frames. The method is based on the structure from motion approach adopted from computer vision, and uses both the intrinsic camera parameters, as well as the tracking transforms associated with each acquired video frame to calculate the global coordinates of the features in the video, and generate a true size 3D model of the imaged scene. We conducted a series of phantom experiments to evaluate the robustness of the proposed method and the accuracy of a generated 3D scene, which yielded 1.7±0.9 mm reconstruction error. We also demonstrated the application of the proposed method using patient-specific endoscopic video image samples acquired during an in vivo gastroscopy procedure.
Augmented Reality for Depth Cues in Monocular Minimally Invasive Surgery

Long Chen,Wen Tang,Nigel W. John,Tao Ruan Wan,Jian Jun Zhang

DOI: https://doi.org/10.48550/arXiv.1703.01243

2017-03-01

Computer Vision and Pattern Recognition

Abstract:One of the major challenges in Minimally Invasive Surgery (MIS) such as laparoscopy is the lack of depth perception. In recent years, laparoscopic scene tracking and surface reconstruction has been a focus of investigation to provide rich additional information to aid the surgical process and compensate for the depth perception issue. However, robust 3D surface reconstruction and augmented reality with depth perception on the reconstructed scene are yet to be reported. This paper presents our work in this area. First, we adopt a state-of-the-art visual simultaneous localization and mapping (SLAM) framework - ORB-SLAM - and extend the algorithm for use in MIS scenes for reliable endoscopic camera tracking and salient point mapping. We then develop a robust global 3D surface reconstruction frame- work based on the sparse point clouds extracted from the SLAM framework. Our approach is to combine an outlier removal filter within a Moving Least Squares smoothing algorithm and then employ Poisson surface reconstruction to obtain smooth surfaces from the unstructured sparse point cloud. Our proposed method has been quantitatively evaluated compared with ground-truth camera trajectories and the organ model surface we used to render the synthetic simulation videos. In vivo laparoscopic videos used in the tests have demonstrated the robustness and accuracy of our proposed framework on both camera tracking and surface reconstruction, illustrating the potential of our algorithm for depth augmentation and depth-corrected augmented reality in MIS with monocular endoscopes.
3D Reconstruction of Unstructured Objects Using Information From Multiple Sensors

Hui Chen,Fangyong Xu,Wanquan Liu,Dongge Sun,Peter Xiaoping Liu,Muhammad Ilyas Menhas,Bilal Ahmad

DOI: https://doi.org/10.1109/jsen.2021.3121343

IF: 4.3

2021-12-01

IEEE Sensors Journal

Abstract:The Structure-from-Motion (SfM) algorithm is widely used for point cloud reconstruction. However, one drawback of conventional SfM based methods is that the obtained final point sets may contain holes and noise, which could degrade the estimation of reconstructed objects especially for smooth surfaces with few features. The other drawback is the accuracy and speed of SfM based methods depend on the uncertain number of images. To overcome these limitations, this paper proposes a novel 3D reconstruction method for unstructured objects based on the structure from motion in combination with the structured light, in which the point sets of structured light and the point sets of structure from motion can come from different target objects. Since the two point sets coming from multiple sensors do not scale well for register, making it difficult to find corresponding points, a scaled principal component analysis algorithm is proposed for the registration to overcome the impact due to large scale variance. With a large scale factor, a recalculated registration center is proposed via feature region segmentation to achieve point cloud registration again. The two point sets are matched using the proposed optimization method to complete 3D reconstruction. Surface reconstruction is performed using the Poisson algorithm to obtain a smooth surface. The proposed method is tested on some simple structured objects and real-life data of complex unstructured objects collected using range sensors. Compared with several state-of-the-art algorithms, experimental results confirm its potential for surface reconstruction from depth data calculated from the two sets.

engineering, electrical & electronic,instruments & instrumentation,physics, applied
Improved shape-from-focus reconstruction for high dynamic range freedom surface

Zhoumiao He,Pei Zhou,Jiangping Zhu,Jianwei Zhang

DOI: https://doi.org/10.1016/j.optlaseng.2023.107784

IF: 5.666

2023-08-25

Optics and Lasers in Engineering

Abstract:The 3D reconstruction of freedom surface has been a major challenge for computer vision as well as metrology. Complex surface structures such as large undulations and deep grooves, along with surface reflection properties like high dynamic range (HDR) pose many difficulties for 3D reconstruction. Shape from focus (SFF) is widely studied for its low cost, ease of deployment, and effectively avoiding occlusion. However, its inherent disadvantages, such as inaccuracy and sensitivity to noise, cannot be ignored. This article demonstrates an improved depth map optimization method to solve these problems. Firstly, a more comprehensive and reliable depth recovery evaluation method is proposed. Then an efficient optimization algorithm is applied to the filtered depth maps. Comparable experiments confirm our method's effective suppression of noise and the reliable compensation of missing signals when measuring freedom surfaces.

optics
An Efficient Dense Reconstruction Algorithm from LiDAR and Monocular Camera

Siyi Xiang,Zepeng Zeng,Jiantao Jiang,Dabo Zhang,Nannan Liu

DOI: https://doi.org/10.3390/sym16111496

2024-11-09

Symmetry

Abstract:Dense reconstruction have been studied for decades in the fields of computer vision and robotics, in which LiDAR and camera are widely used. However, vision-based methods are sensitive to illumination variation and lack direct depth, and LiDAR-based methods are limited by sparse LiDAR measurement and lacking color and texture information. In this paper, we propose a novel 3D reconstruction algorithm based on LiDAR and a monocular camera, which realizes dense reconstruction. In the algorithm, a LiDAR odometry is used to get accurate poses and poses calculated by the odometry module are used in the calculation of depth maps and fusion of depth maps, and then mesh and texture mapping are implemented. In addition, a semantic segmentation network and a depth completion network are used to obtain dense and accurate depth maps. The concept of symmetry is utilized to generate 3D models of objects or scenes; that is, the reconstruction and camera imaging of these objects or scenes are symmetrical. Experimental results on public dataset show that the proposed algorithm achieves higher accuracy, efficiency and completeness than existing methods.

multidisciplinary sciences
High-quality texture reconstruction for three-dimensional multi-view imaging (Conference Presentation)

Jiachen Wu,Liangcai Cao,Guofan Jin

DOI: https://doi.org/10.1117/12.2509615

2019-01-01

Abstract:Three-dimensional (3D) model with high-quality texture is a powerful way to interact between virtual and reality 3D scene. It has significant applications in digital preservation of cultural heritage, virtual reality / augmented reality (VR/AR), medical imaging and other domains. High-quality texture reconstruction is one of the essential elements for 3D model. Due to the camera pose error and inaccurate model, current texture reconstruction methods could not eliminate the artifacts like blurring, ghosting and discontinuity. In this work, multi-view high-definition images are captured for texture mapping. Normal-weight, depth-weight and edge-weight parameter are introduced to evaluate texture color confidence, respectively. The normalized weight factors are multiplied to generate a comprehensive weight parameter. By taking a weighted average of projected texture images, discontinuity of texture can be smoothed to a large extent. For large misalignment, bidirectional similarity (BDS) function, which represent the structural similarity between two images, are utilized to improve the texture image. The energy function is composed of two terms. One is Euclidean distance between target texture image and merged texture image. The other is BDS between target texture image and original texture image. By minimizing the energy function, the texture image could generate small local displacement while retaining the original structural information. The target and merged images are optimized alternately, the target image is calculated by patch-match algorithm, and the merged image is derived from weighted average of target images. The method we proposed could produce seamless texture comparing with Markov random field (MRF) algorithm. The definition could be higher than the camera parameter optimization algorithm. The experimental results show that structural similarity (SSIM) between the reconstructed images and the ground truth is higher than traditional algorithms. When dealing with inaccurate models with less than 10,000 facets, the SSIM value could be doubled compared with current algorithms.
Surface Reconstruction from Sparse and Mutually Intersected Contours for Freehand 3D Ultrasound Using Variational Method

Shuangcheng Deng,Yunhua Li,Lipei Jiang,Yingyu Cao,Junwen Zhang

DOI: https://doi.org/10.1109/isdea.2012.653

2012-01-01

Abstract:The 3D reconstruction for freehand 3D ultrasound is a challenging issue because the recorded B-scans are not only sparse, but also non-parallel (actually they may intersect each other). Conventional volume reconstruction methods can't reconstruct sparse data efficiently while not introducing geometrical artifacts, and conventional surface reconstruction methods can't reconstruct surfaces from contours that are arbitrarily oriented in 3D space. We developed a new surface reconstruction method for freehand 3D ultrasound. It is based on variational implicit function which is presented by Greg Turk for shape transformation. In the new method, we first constructed on-& off-surface constraints from the segmented contours of all recorded B-scans, then used a variational interpolation technique to get a single implicit function in 3D. Finally, the implicit function was evaluated to extract the zero-valued surface as reconstruction result. Two experiment was conducted to assess our variational surface reconstruction method, and the experiment results have shown that the new method is capable of reconstructing surface smoothly from sparse contours which can be arbitrarily oriented in 3D space.
Real-time Dense Reconstruction of Tissue Surface from Stereo Optical Video

Haoyin Zhou,Jagadeesan Jayender

DOI: https://doi.org/10.48550/arXiv.2007.12623

2020-07-17

Abstract:We propose an approach to reconstruct dense three-dimensional (3D) model of tissue surface from stereo optical videos in real-time, the basic idea of which is to first extract 3D information from video frames by using stereo matching, and then to mosaic the reconstructed 3D models. To handle the common low texture regions on tissue surfaces, we propose effective post-processing steps for the local stereo matching method to enlarge the radius of constraint, which include outliers removal, hole filling and smoothing. Since the tissue models obtained by stereo matching are limited to the field of view of the imaging modality, we propose a model mosaicking method by using a novel feature-based simultaneously localization and mapping (SLAM) method to align the models. Low texture regions and the varying illumination condition may lead to a large percentage of feature matching outliers. To solve this problem, we propose several algorithms to improve the robustness of SLAM, which mainly include (1) a histogram voting-based method to roughly select possible inliers from the feature matching results, (2) a novel 1-point RANSAC-based P$n$P algorithm called as DynamicR1PP$n$P to track the camera motion and (3) a GPU-based iterative closest points (ICP) and bundle adjustment (BA) method to refine the camera motion estimation results. Experimental results on ex- and in vivo data showed that the reconstructed 3D models have high resolution texture with an accuracy error of less than 2 mm. Most algorithms are highly parallelized for GPU computation, and the average runtime for processing one key frame is 76.3 ms on stereo images with 960x540 resolution.

Computer Vision and Pattern Recognition
2D-D Lifting for Shape Reconstruction

Liangliang Nan,Andrei Sharf,Baoquan Chen

DOI: https://doi.org/10.1111/cgf.12493

IF: 2.5

2014-01-01

Computer Graphics Forum

Abstract:We present an algorithm for shape reconstruction from incomplete 3D scans by fusing together two acquisition modes: 2D photographs and 3D scans. The two modes exhibit complementary characteristics: scans have depth information, but are often sparse and incomplete; photographs, on the other hand, are dense and have high resolution, but lack important depth information. In this work we fuse the two modes, taking advantage of their complementary information, to enhance 3D shape reconstruction from an incomplete scan with a 2D photograph. We compute geometrical and topological shape properties in 2D photographs and use them to reconstruct a shape from an incomplete 3D scan in a principled manner. Our key observation is that shape properties such as boundaries, smooth patches and local connectivity, can be inferred with high confidence from 2D photographs. Thus, we register the 3D scan with the 2D photograph and use scanned points as 3D depth cues for lifting 2D shape structures into 3D. Our contribution is an algorithm which significantly regularizes and enhances the problem of 3D reconstruction from partial scans by lifting 2D shape structures into 3D. We evaluate our algorithm on various shapes which are loosely scanned and photographed from different views, and compare them with state-of-the-art reconstruction methods.

Phase II study of 1,2,4-triglycidylurazol (TGU) in previously untreated and treated patients with small cell lung cancer.

A clustering approach to free form surface reconstruction from multi-view range images

High-fidelity 3D Face Reconstruction with Multi-Scale Details

Prior surface model guided dense reconstruction via graph-cuts

High-Quality Textured 3D Shape Reconstruction with Cascaded Fully Convolutional Networks

Enhanced 3D Shape Reconstruction With Knowledge Graph of Category Concept

Geometry-aware Two-scale PIFu Representation for Human Reconstruction

Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture

Surface Reconstruction from Sparse & Arbitrarily Oriented Contours in Freehand 3D Ultrasound.

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Variational Approach To Reconstruct Surface From Sparse & Nonparallel Contours In Freehand 3d Ultrasound Imaging

Toward a Clinically‐Oriented Model of Anxiety Disorders

Surface Reconstruction from Tracked Endoscopic Video Using the Structure from Motion Approach.

Augmented Reality for Depth Cues in Monocular Minimally Invasive Surgery

3D Reconstruction of Unstructured Objects Using Information From Multiple Sensors

Improved shape-from-focus reconstruction for high dynamic range freedom surface

An Efficient Dense Reconstruction Algorithm from LiDAR and Monocular Camera

High-quality texture reconstruction for three-dimensional multi-view imaging (Conference Presentation)

Surface Reconstruction from Sparse and Mutually Intersected Contours for Freehand 3D Ultrasound Using Variational Method

Real-time Dense Reconstruction of Tissue Surface from Stereo Optical Video

2D-D Lifting for Shape Reconstruction