5-1-2005 Bias Affiliated With Two Variants Of Cohen ’ s d When Determining U 1 As A Measure Of The Percent Of Non-Overlap

D. Walker

Abstract:

What problem does this paper attempt to address?

Highlight Removal for Endoscopic Images based on Accelerated Adaptive Non-Convex RPCA Decomposition

Junjun Pan,Ranyang Li,Hongjun Liu,Yong Hu,Wenhao Zheng,Bin Yan,Yunsheng Yang,Yi Xiao

DOI: https://doi.org/10.1016/j.cmpb.2022.107240

IF: 6.1

2022-11-18

Computer Methods and Programs in Biomedicine

Abstract:Objective: Highlights always occur in endoscopic images due to their special imaging environment. It not only increases the difficulty of observation and diagnosis from surgeons, but also influences the performance of Mixed/Augmented Reality techniques in surgery navigation. Methods: In this paper, we propose a novel accelerated adaptive non-convex robust principal component analysis method (AANC-RPCA) to remove highlights in endoscopic images. We first detect absolute highlight pixels of the enhanced endoscopic images with adaptive threshold segmentation. The quasi-convex function is proposed to approximate a new non-convex objective function. With detected highlight pixels and quasi-convex function, it introduces gradient to shrink sparse matrix and obtains a faster speed of convergence. Then we divide the image into multiple blocks and perform the parallel computation to enhance the efficiency. Finally, we design a weighted template that decays outward with dilation and linear filtering to reconstruct the endoscopic images. Our approach is almost independent of hyper-parameters and can achieve adaptive decomposition. Results: It has been verified on multiple types of endoscopic images through experiments and clinical blind tests. The results demonstrate that our method can obtain the best performance for the recovered images with more details in a shorter time (about 3-5 times). Conclusion: Coupled with the user study, both the quantitative and qualitative results indicate that our approach has the potential to be highly useful in endoscopy images. Compared with the existing highlight removal approaches, our method obtains the SOTA results and has the potential to be applied in the various medical processing processes.

engineering, biomedical,computer science, interdisciplinary applications,medical informatics, theory & methods
Parameter-Free Matrix Decomposition for Specular Reflections Removal in Endoscopic Images

Jithin Joseph,Sudhish N George,Kiran Raja

DOI: https://doi.org/10.1109/JTEHM.2023.3283444

2023-06-06

Abstract:Objective: Endoscopy is a medical diagnostic procedure used to see inside the human body with the help of a camera-attached system called the endoscope. Endoscopic images and videos suffer from specular reflections (or highlight) and can have an adverse impact on the diagnostic quality of images. These scattered white regions severely affect the visual appearance of images for both endoscopists and the computer-aided diagnosis of diseases. Methods & Results: We introduce a new parameter-free matrix decomposition technique to remove the specular reflections. The proposed method decomposes the original image into a highlight-free pseudo-low-rank component and a highlight component. Along with the highlight removal, the approach also removes the boundary artifacts present around the highlight regions, unlike the previous works based on family of Robust Principal Component Analysis (RPCA). The approach is evaluated on three publicly available endoscopy datasets: Kvasir Polyp, Kvasir Normal-Pylorus and Kvasir Capsule datasets. Our evaluation is benchmarked against 4 different state-of-the-art approaches using three different well-used metrics such as Structural Similarity Index Measure (SSIM), Percentage of highlights remaining and Coefficient of Variation (CoV). Conclusions: The results show significant improvements over the compared methods on all three metrics. The approach is further validated for statistical significance where it emerges better than other state-of-the-art approaches.Clinical and Translational Impact Statement-The mathematical concepts of low rank and rank decomposition in matrix algebra are translated to remove specularities in the endoscopic images The result shows the impact of the proposed method in removing specular reflections from endoscopic images indicating improved diagnosis efficiency for both endoscopists and computer-aided diagnosis systems.
Specular Reflections Detection and Removal for Endoscopic Images Based on Brightness Classification

Chao Nie,Chao Xu,Zhengping Li,Lingling Chu,Yunxue Hu

DOI: https://doi.org/10.3390/s23020974

2023-01-14

Abstract:Specular Reflections often exist in the endoscopic image, which not only hurts many computer vision algorithms but also seriously interferes with the observation and judgment of the surgeon. The information behind the recovery specular reflection areas is a necessary pre-processing step in medical image analysis and application. The existing highlight detection method is usually only suitable for medium-brightness images. The existing highlight removal method is only applicable to images without large specular regions, when dealing with high-resolution medical images with complex texture information, not only does it have a poor recovery effect, but the algorithm operation efficiency is also low. To overcome these limitations, this paper proposes a specular reflection detection and removal method for endoscopic images based on brightness classification. It can effectively detect the specular regions in endoscopic images of different brightness and can improve the operating efficiency of the algorithm while restoring the texture structure information of the high-resolution image. In addition to achieving image brightness classification and enhancing the brightness component of low-brightness images, this method also includes two new steps: In the highlight detection phase, the adaptive threshold function that changes with the brightness of the image is used to detect absolute highlights. During the highlight recovery phase, the priority function of the exemplar-based image inpainting algorithm was modified to ensure reasonable and correct repairs. At the same time, local priority computing and adaptive local search strategies were used to improve algorithm efficiency and reduce error matching. The experimental results show that compared with the other state-of-the-art, our method shows better performance in terms of qualitative and quantitative evaluations, and the algorithm efficiency is greatly improved when processing high-resolution endoscopy images.
Augmented Reality Computational Spectral Imaging for Surgical Guidance

Tao Chenning,Zheng Zhenrong

DOI: https://doi.org/10.3788/lop202259.2011014

2022-01-01

Laser & Optoelectronics Progress

Abstract:Surgical guidance based on spectral imaging can effectively identify various tissue types by analyzing the spectrum differences of different tissues, which has a significant application value. However, the current sampling speed of spectral imaging and the presentation of spectral images severely limit its clinical applications. This paper proposes an augmented reality (AR) computational spectral imaging system for facilitating surgical guidance. High-quality spectral reconstruction is achieved in a single detection using RGB imaging devices and multispectral single-pixel detectors, which greatly improves the imaging speed and reconstruction efficiency. Principal component analysis and spectral angle mapping are adopted to extract the effective information of the spectral images, which are then used to highlight the characteristic tissue area. The image enhancement effect based on the fusion of the spectral image and actual surgical area is demonstrated in the head-mounted AR display device.
EndoSRR: a comprehensive multi-stage approach for endoscopic specular reflection removal

Wei Li,Fucang Jia,Wenjian Liu

DOI: https://doi.org/10.1007/s11548-024-03137-8

2024-04-22

International Journal of Computer Assisted Radiology and Surgery

Abstract:Specular reflections in endoscopic images not only disturb visual perception but also hamper computer vision algorithm performance. However, the intricate nature and variability of these reflections, coupled with a lack of relevant datasets, pose ongoing challenges for removal.

engineering, biomedical,radiology, nuclear medicine & medical imaging,surgery
Accurate and robust feature description and dense point-wise matching based on feature fusion for endoscopic images

Ranyang Li,Junjun Pan,Yongming Yang,Nan Wei,Bin Yan,Hao Liu,Yunsheng Yang,Hong Qin

DOI: https://doi.org/10.1016/j.compmedimag.2021.102007

IF: 7.422

2021-12-01

Computerized Medical Imaging and Graphics

Abstract:Despite the rapid technical advancement of augmented reality (AR) and mixed reality (MR) in minimally invasive surgery (MIS) in recent years, monocular-based 2D/3D reconstruction still remains technically challenging in AR/MR guided surgery navigation nowadays. In principle, soft tissue surface is smooth and watery with sparse texture, specular reflection, and frequent deformation. As a result, we frequently obtain only sparse feature points that give rise to incorrect matching results with conventional image processing methods. To ameliorate, in this paper we enunciate an accurate and robust description and matching method for dense feature points in endoscopic videos. Our new method first extracts contours of the low-rank image sequences based on the adaptive robust principal component analysis (RPCA) decomposition. Then we propose a multi-scale dense geometric feature description approach, which simultaneously extracts dense feature descriptors of the contours in the original Euclidean coordinate space, the accompanying 3D color coordinate space, and the derived curvature-gradient coordinate space. Finally, we devise a new algorithm for both global and local point-wise matching based on feature fusion. For global matching, we employ the fast Fourier transform (FFT) to reduce the dimension of the dense feature descriptors. For local feature point matching, in order to enhance the robustness and accuracy of the matching, we cluster multiple contour points to form "super-point" based on dense feature descriptors and their spatio-temporal continuity. The comprehensive experimental results confirm that our novel approach can overcome the highlight influence, and robustly describe contours from image sequences of soft tissue surfaces. Compared with the state-of-the-art feature point description and matching methods, our analysis framework shows the key advantages of both robustness and accuracy in dense point-wise matching, even when the severe soft tissue deformation occurs. Our new approach is expected to have high potential in 2D/3D reconstruction in endoscopy.

engineering, biomedical,radiology, nuclear medicine & medical imaging
Specular highlight removal for endoscopic images using partial attention network

Chong Zhang,Yueliang Liu,Kun Wang,Jie Tian

DOI: https://doi.org/10.1088/1361-6560/ad02d9

IF: 3.5

2023-10-14

Physics in Medicine and Biology

Abstract:Objective. Endoscopic imaging is a visualization method widely used in minimally invasive surgery. However, owing to the strong reflection of the mucus layer on the organs, specular highlights often appear to degrade the imaging performance. Thus, it is necessary to develop an effective highlight removal method for endoscopic imaging. Approach. A specular highlight removal method using a partial attention network (PatNet) for endoscopic imaging is proposed to reduce the interference of bright light in endoscopic surgery. The method is designed as two procedures: highlight segmentation and endoscopic image inpainting. Image segmentation uses brightness threshold based on illumination compensation to divide the endoscopic image into the highlighted mask and the non-highlighted area. The image inpainting algorithm uses a partial convolution network that integrates an attention mechanism. A mask dataset with random hopping points is designed to simulate specular highlight in endoscopic imaging for network training. Through the filtering of masks, the method can focus on recovering defective pixels and preserving valid pixels as much as possible. Main results. The PatNet is compared with 3 highlight segmentation methods, 3 imaging inpainting methods and 5 highlight removal methods for effective analysis. Experimental results show that the proposed method provides better performance in terms of both perception and quantification. In addition, surgeons are invited to score the processing results for different highlight removal methods under realistic reflection conditions. The PatNet received the highest score of 4.18. Correspondingly, the kendall's W is 0.757 and the asymptotic significance p=0.000<0.01, revealing that the subjective scores have good consistency and confidence. Significance. Generally, the method can realize irregular shape highlight reflection removal and image restoration close to the ground truth of endoscopic images. This method can improve the quality of endoscopic imaging for accurate image analysis.

engineering, biomedical,radiology, nuclear medicine & medical imaging
PAR<inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math><alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msup><mml:mrow/><mml:mn>2</mml:mn></mml:msup></mml:math><inline-graphic xlink:href="hong-ieq1-3286429.gif" xmlns:xlink="http://www.w3.org/1999/xlink"/></alternatives></inline-formula>Net: End-to-End Panoramic Image Reflection Removal

Yuchen Hong,Qian Zheng,Lingran Zhao,Xudong Jiang,Alex C. Kot,Boxin Shi

DOI: https://doi.org/10.1109/TPAMI.2023.3286429

IF: 23.6

2023-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:In this article, we investigate the problem of panoramic image reflection removal to relieve the content ambiguity between the reflection layer and the transmission scene. Although a partial view of the reflection scene is attainable in the panoramic image and provides additional information for reflection removal, it is not trivial to directly apply this for getting rid of undesired reflections due to its misalignment with the reflection-contaminated image. We propose an end-to-end framework to tackle this problem. By resolving misalignment issues with adaptive modules, the high-fidelity recovery of reflection layer and transmission scenes is accomplished. We further propose a new data generation approach that considers the physics-based formation model of mixture images and the in-camera dynamic range clipping to diminish the domain gap between synthetic and real data. Experimental results demonstrate the effectiveness of the proposed method and its applicability for mobile devices and industrial applications.
Specular Highlight Detection Based on Color Distribution for Endoscopic Images

Baoxian Yu,Wanbing Chen,Qinghua Zhong,Han Zhang

DOI: https://doi.org/10.3389/fphy.2020.616930

IF: 3.718

2021-01-12

Frontiers in Physics

Abstract:Endoscopic imaging systems have been widely used in disease diagnosis and minimally invasive surgery. Practically, specular reflection (a.k.a. highlight) always exists in endoscopic images and significantly affects surgeons’ observation and judgment. Motivated by the fact that the values of the red channel in nonhighlight area of endoscopic images are higher than those of the green and blue ones, this paper proposes an adaptive specular highlight detection method for endoscopic images. Specifically, for each pixel, we design a criterion for specular highlight detection based on the ratio of the red channel to both the green and blue channels. With the designed criteria, we take advantage of image segmentation and then develop an adaptive threshold with respect to the differences between the red channel and the other ones of neighboring pixels. To validate the proposed method, we conduct experiments on clinical data and CVC-ClinicSpec open database. The experimental results demonstrate that the proposed method yields an averaged Precision , Accuracy , and F 1 -score rate of 88.76%, 99.60% and 72.56%, respectively, and outperforms the state-of-the-art approaches based on color distribution reported for endoscopic highlight detection.

physics, multidisciplinary
EasySpec: Automatic Specular Reflection Detection and Suppression From Endoscopic Images

Wenjun Shan,P. Monkam,Hao Chen,Yuhao Zhai,Jing Wu,Wen-kai Lu

DOI: https://doi.org/10.1109/TCI.2021.3112117

IF: 5.4

IEEE Transactions on Computational Imaging

Abstract:The outcome of endoscopic tasks can be significantly affected by the presence of specular reflections. Although numerous methods have been proposed for specular reflection detection and suppression from endoscopic images, they are inefficient, usually require tedious empirical parameters selection, and perform poorly when handling large specular regions. To this end, we propose a robust and efficient deep learning framework termed EasySpec for identifying and suppressing specular reflections from endoscopic images. Our proposed EasySpec consists of two stages: a detection stage and a suppression stage. The former stage is performed using a lightweight UNet-variant dubbed Scaled-UNet, which is trained exploiting a novel hybrid strategy that integrates the advantages of both transfer learning and weakly supervised techniques. The latter stage is achieved utilizing the concept of deep image inpainting. Specifically, a new and fast end-to-end approach for inpainting multi-size specular reflection regions in endoscopic images is developed. Our proposed approach termed GatedResUNet, takes advantage of the gated convolution to accurately differentiate specular pixels from non-specular pixels, and the U-Net architecture to learn multi-level semantic representative features, which helps reconstruct more realistic and semantically plausible images. Extensive qualitative and quantitative experiments on several endoscopic datasets acquired from different MIS scenes reveal that our proposed EasySpec outperforms state-of-the-art endoscopic specular reflection suppression approaches. Specifically, EasySpec generalizes well on various MIS scenes, and can properly reconstruct blood vessels and fine details in a reasonable processing time, which validates its practical significance.

Engineering,Medicine,Computer Science
Sudden cardiac death due to anomalous origin of the right coronary artery: a case report in a child.

V. Fineschi,E. Maresi,M. Di Padua,I. Riezzo,M. Neri

DOI: https://doi.org/10.1016/J.IJCARD.2005.03.076

IF: 4.039

2006-04-14

International Journal of Cardiology

Abstract:
A Temporal Learning Approach to Inpainting Endoscopic Specularities and Its effect on Image Correspondence

Rema Daher,Francisco Vasconcelos,Danail Stoyanov

DOI: https://doi.org/10.48550/arXiv.2203.17013

2022-03-31

Abstract:Video streams are utilised to guide minimally-invasive surgery and diagnostic procedures in a wide range of procedures, and many computer assisted techniques have been developed to automatically analyse them. These approaches can provide additional information to the surgeon such as lesion detection, instrument navigation, or anatomy 3D shape modeling. However, the necessary image features to recognise these patterns are not always reliably detected due to the presence of irregular light patterns such as specular highlight reflections. In this paper, we aim at removing specular highlights from endoscopic videos using machine learning. We propose using a temporal generative adversarial network (GAN) to inpaint the hidden anatomy under specularities, inferring its appearance spatially and from neighbouring frames where they are not present in the same location. This is achieved using in-vivo data of gastric endoscopy (Hyper-Kvasir) in a fully unsupervised manner that relies on automatic detection of specular highlights. System evaluations show significant improvements to traditional methods through direct comparison as well as other machine learning techniques through an ablation study that depicts the importance of the network's temporal and transfer learning components. The generalizability of our system to different surgical setups and procedures was also evaluated qualitatively on in-vivo data of gastric endoscopy and ex-vivo porcine data (SERV-CT, SCARED). We also assess the effect of our method in computer vision tasks that underpin 3D reconstruction and camera motion estimation, namely stereo disparity, optical flow, and sparse point feature matching. These are evaluated quantitatively and qualitatively and results show a positive effect of specular highlight inpainting on these tasks in a novel comprehensive analysis.

Computer Vision and Pattern Recognition
The use of tranexamic acid to reduce postoperative bleeding following cardiac surgery: a double-blind randomized trial.

Arthur C. Coffey,J. Pittmam,H. Halbrook,J. Fehrenbacher,D. Beckman,D. Hormuth

1995-07-01

The American Surgeon

Abstract:Bleeding during the first 24 hours following cardiac surgery using cardio-pulmonary bypass (CPB) is a serious complication. Attempts to modify the degree of postoperative bleeding with pharmacologic therapy have met with limited success. Tranexamic acid, a potent inhibitor of plasminogen, may decrease the amount of mediastinal bleeding following surgery utilizing CPB. We assigned 30 patients who were about to undergo cardiac surgery to a double-blind randomized trial. The treatment group received tranexamic acid, given intravenously as 10 mg/kg over 30 minutes, at the time of skin incision, followed by a 1 mg/kg/hr infusion for 12 hours. The control group received a placebo (saline) of equal volume. Measurements of shed mediastinal blood and transfused homologous blood were made at 6, 12, and 24 hours postoperatively. Mediastinal blood loss in the treatment and control groups was 382 mL versus 594 mL at 6 hours (P = 0.08), 502 mL versus 848 mL at 12 hours (P = 0.04), and 711 mL versus 1160 mL at 24 hours (P = 0.02). The mean transfusion volumes after 24 hours were 356 mL in the treatment group and 528 mL in the placebo group (P = NS). Prophylactic tranexamic acid infusion decreases mediastinal blood loss following cardiopulmonary assisted cardiac surgery.
Dynamic obstruction of the left main coronary artery ostium by a papillary fibroelastoma.

L. C. Napp,H. Baraki,I. Kutschka,M. Bredt,M. Brehm,J. Bauersachs,U. Bavendiek

DOI: https://doi.org/10.1016/j.carpath.2013.05.004

IF: 3.975

Cardiovascular Pathology

Abstract:
A Hessian-Based Technique for Specular Reflection Detection and Inpainting in Colonoscopy Images

Khaled ELKarazle,Valliappan Raman,Caslon Chua,Patrick Then

DOI: https://doi.org/10.1109/JBHI.2024.3404955

Abstract:In the field of Computer-Aided Detection (CADx), the use of AI-based algorithms for disease detection in endoscopy images, especially colonoscopy images, is on the rise. However, these algorithms often encounter performance issues due to obstructions like specular reflection, resulting in false positives. This paper presents a novel algorithm specifically designed to tackle the challenges posed by high specular reflection regions in colonoscopy images. The proposed algorithm identifies these regions and applies precise inpainting for restoration. The process entails converting the input image from RGB to HSV color space and focusing on the Saturation (S) component in convex regions detected using a Hessian-based method. This step creates a binary mask that pinpoints areas of specular reflection. The inpainting function then uses this mask to guide the restoration of these identified regions and their borders. To ensure a seamless blend of the restored regions with the background and adjacent pixels, a feathering process is applied to the repaired regions. This enhances both the accuracy and aesthetic coherence of the inpainted images. The performance of our algorithm was rigorously tested on five unique colonoscopy datasets and various endoscopy images from the Kvasir dataset, using an extensive set of evaluation metrics and a comparative analysis with existing methods consistently highlighted the superior performance of our algorithm.
Relationship between the levels of cyclic cytidine 3':5'-monophosphate, cyclic guanosine 3':5'-monophosphate, and cyclic adenosine 3':5'-monophosphate in urines and leukocytes and the type of human leukemias.

J. Scavennec,Y. Carcassonne,J. Gastaut,A. Blanc,H. Cailla

IF: 11.2

1981-08-01

Cancer Research

Abstract:Cyclic cytidine 3':5'-monophosphate (cyclic CMP), cyclic guanosine 3':5'-monophosphate (cyclic GMP), and cyclic adenosine 3':5'-monophosphate (cyclic AMP) contents of leukocytes and urines of leukemic patients have been investigated. We have studied four types of leukemia: acute myeloblastic leukemia; chronic myelocytic leukemia; acute lymphoblastic leukemia; and chronic lymphocytic leukemia. As controls, the cyclic nucleotide content of leukocytes and urines of healthy volunteers and patients with solid tumors selected for their normal hemogram has been determined. It has also been measured in phytohemagglutinin-stimulated lymphocytes. Our data show that: (a) the concentration of cyclic CMP is always lower than that of cyclic GMP or cyclic AMP; (b) in urines, the concentrations of the three nucleotides are higher in patients than in healthy volunteers, the greatest differences being observed between the cyclic CMP concentrations of acute leukemia patients and controls; and (c) in white blood cells, cyclic AMP concentration is lower in leukemic than in normal cells. The cyclic GMP concentration is the same everywhere except in monoblastic cells and leukocytes from solid tumor patients. High cyclic CMP levels are associated only with acute leukemia, whether myeloblastic, monoblastic, or lymphoblastic, a fact which suggests that cyclic CMP could be a biochemical marker of hematopoietic stem cell malignancy.
Speckle reduction in laser illuminated endoscopy using adversarial deep learning

Taylor L. Bobrow,Faisal Mahmood,Miguel Inserni,Nicholas J. Durr

DOI: https://doi.org/10.1117/12.2510587

2019-03-04

Abstract:Endoscope size is a major design constraint that must be managed with the clinical demand for high-quality illumination and imaging. Existing commercial endoscopes most often use an arc lamp to produce bright, incoherent white light, requiring large-area fiber bundles to deliver sufficient illumination power to the sample. Moreover, the power instability of these light sources creates challenges for computer vision applications. We demonstrate an alternative illumination technique using red-green-blue laser light and a data-driven approach to combat the speckle noise that is a byproduct of coherent illumination. We frame the speckle artifact problem as an image-to-image translation task solved using conditional Generative Adversarial Networks (cGANs). To train the network, we acquire images illuminated with a coherent laser diode, with a laser diode source made partially- coherent using a laser speckle reducer, and with an incoherent LED light source as the target domain. We train networks using laser-illuminated endoscopic images of ex-vivo, porcine gastrointestinal tissues, augmented by images of laser-illuminated household and laboratory objects. The network is then benchmarked against state of-the-art optical and image processing speckle reduction methods, achieving an increased peak signal-to-noise ratio (PSNR) of 4.1 db, compared to 0.7 dB using optical speckle reduction, 0.6 dB using median filtering, and 0.5 dB using non-local means. This approach not only allows for endoscopes with smaller, more efficient light sources with extremely short triggering times, but it also enables imaging modalities that require both coherent and incoherent sources, such as combined widefield and speckle ow contrast imaging in a single image frame.
PR-ENDO: Physically Based Relightable Gaussian Splatting for Endoscopy

Joanna Kaleta,Weronika Smolak-Dyżewska,Dawid Malarz,Diego Dall'Alba,Przemysław Korzeniowski,Przemysław Spurek

2024-11-19

Abstract:Endoscopic procedures are crucial for colorectal cancer diagnosis, and three-dimensional reconstruction of the environment for real-time novel-view synthesis can significantly enhance diagnosis. We present PR-ENDO, a framework that leverages 3D Gaussian Splatting within a physically based, relightable model tailored for the complex acquisition conditions in endoscopy, such as restricted camera rotations and strong view-dependent illumination. By exploiting the connection between the camera and light source, our approach introduces a relighting model to capture the intricate interactions between light and tissue using physically based rendering and MLP. Existing methods often produce artifacts and inconsistencies under these conditions, which PR-ENDO overcomes by incorporating a specialized diffuse MLP that utilizes light angles and normal vectors, achieving stable reconstructions even with limited training camera rotations. We benchmarked our framework using a publicly available dataset and a newly introduced dataset with wider camera rotations. Our methods demonstrated superior image quality compared to baseline approaches.

Computer Vision and Pattern Recognition
Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields

Zhibao Qin,Qi Chen,Kai Qian,Qinhong Zheng,Junsheng Shi,Yonghang Tai

DOI: https://doi.org/10.1364/boe.521612

IF: 3.4

2024-05-23

Biomedical Optics Express

Abstract:Virtual surgical training is crucial for enhancing minimally invasive surgical skills. Traditional geometric reconstruction methods based on medical CT/MRI images often fall short in providing color information, which is typically generated through pseudo-coloring or artistic rendering. To simultaneously reconstruct both the geometric shape and appearance information of organs, we propose a novel organ model reconstruction network called Endoscope-NeSRF. This network jointly leverages neural radiance fields and Signed Distance Function (SDF) to reconstruct a textured geometric model of the organ of interest from multi-view photometric images acquired by an endoscope. The prior knowledge of the inverse correlation between the distance from the light source to the object and the radiance improves the real physical properties of the organ. The dilated mask further refines the appearance and geometry at the organ's edges. We also proposed a highlight adaptive optimization strategy to remove highlights caused by the light source during the acquisition process, thereby preventing the reconstruction results in areas previously affected by highlights from turning white. Finally, the real-time realistic rendering of the organ model is achieved by combining the inverse rendering and Bidirectional Reflectance Distribution Function (BRDF) rendering methods. Experimental results show that our method closely matches the Instant-NGP method in appearance reconstruction, outperforming other state-of-the-art methods, and stands as the superior method in terms of geometric reconstruction. Our method obtained a detailed geometric model and realistic appearance, providing a realistic visual sense for virtual surgical simulation, which is important for medical training.

radiology, nuclear medicine & medical imaging,biochemical research methods,optics

5-1-2005 Bias Affiliated With Two Variants Of Cohen ’ s d When Determining U 1 As A Measure Of The Percent Of Non-Overlap

Highlight Removal for Endoscopic Images based on Accelerated Adaptive Non-Convex RPCA Decomposition

Parameter-Free Matrix Decomposition for Specular Reflections Removal in Endoscopic Images

Specular Reflections Detection and Removal for Endoscopic Images Based on Brightness Classification

Augmented Reality Computational Spectral Imaging for Surgical Guidance

EndoSRR: a comprehensive multi-stage approach for endoscopic specular reflection removal

Accurate and robust feature description and dense point-wise matching based on feature fusion for endoscopic images

Specular highlight removal for endoscopic images using partial attention network

Specular Highlight Detection Based on Color Distribution for Endoscopic Images

EasySpec: Automatic Specular Reflection Detection and Suppression From Endoscopic Images

Sudden cardiac death due to anomalous origin of the right coronary artery: a case report in a child.

A Temporal Learning Approach to Inpainting Endoscopic Specularities and Its effect on Image Correspondence

The use of tranexamic acid to reduce postoperative bleeding following cardiac surgery: a double-blind randomized trial.

Dynamic obstruction of the left main coronary artery ostium by a papillary fibroelastoma.

A Hessian-Based Technique for Specular Reflection Detection and Inpainting in Colonoscopy Images

Relationship between the levels of cyclic cytidine 3':5'-monophosphate, cyclic guanosine 3':5'-monophosphate, and cyclic adenosine 3':5'-monophosphate in urines and leukocytes and the type of human leukemias.

Speckle reduction in laser illuminated endoscopy using adversarial deep learning

PR-ENDO: Physically Based Relightable Gaussian Splatting for Endoscopy

Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields