Cognitive rehearsal in the treatment of obesity: a comparison against cue avoidance and social pressure.

G. Bennett

DOI: https://doi.org/10.1016/0306-4603(86)90051-1

Abstract:

What problem does this paper attempt to address?

Towards Robotic Eye Surgery: Marker-free, Online Hand-eye Calibration using Optical Coherence Tomography Images

Mingchuan Zhou,Mahdi Hamad,Jakob Weiss,Abouzar Eslami,Kai Huang,Mathias Maier,Chris P. Lohmann,Nassir Navab,Alois Knoll,M. Ali Nasseri

DOI: https://doi.org/10.1109/LRA.2018.2858744

2018-08-17

Abstract:Ophthalmic microsurgery is known to be a challenging operation, which requires very precise and dexterous manipulation. Image guided robot-assisted surgery (RAS) is a promising solution that brings significant improvements in outcomes and reduces the physical limitations of human surgeons. However, this technology must be further developed before it can be routinely used in clinics. One of the problems is the lack of proper calibration between the robotic manipulator and appropriate imaging device. In this work, we developed a flexible framework for hand-eye calibration of an ophthalmic robot with a microscope-integrated Optical Coherence Tomography (MIOCT) without any markers. The proposed method consists of three main steps: a) we estimate the OCT calibration parameters; b) with micro-scale displacements controlled by the robot, we detect and segment the needle tip in 3D-OCT volume; c) we find the transformation between the coordinate system of the OCT camera and the coordinate system of the robot. We verified the capability of our framework in ex-vivo pig eye experiments and compared the results with a reference method (marker-based). In all experiments, our method showed a small difference from the marker based method, with a mean calibration error of 9.2 $\mu$m and 7.0 $\mu$m, respectively. Additionally, the noise test shows the robustness of the proposed method.

Robotics,Computer Vision and Pattern Recognition
Augmented reality calibration using feature triangulation iteration-based registration for surgical navigation

Long Shao,Shuo Yang,Tianyu Fu,Yucong Lin,Haixiao Geng,Danni Ai,Jingfan Fan,Hong Song,Tao Zhang,Jian Yang

DOI: https://doi.org/10.1016/j.compbiomed.2022.105826

Abstract:Background: Marker-based augmented reality (AR) calibration methods for surgical navigation often require a second computed tomography scan of the patient, and their clinical application is limited due to high manufacturing costs and low accuracy. Methods: This work introduces a novel type of AR calibration framework that combines a Microsoft HoloLens device with a single camera registration module for surgical navigation. A camera is used to gather multi-view images of a patient for reconstruction in this framework. A shape feature matching-based search method is proposed to adjust the size of the reconstructed model. The double clustering-based 3D point cloud segmentation method and 3D line segment detection method are also proposed to extract the corner points of the image marker. The corner points are the registration data of the image marker. A feature triangulation iteration-based registration method is proposed to quickly and accurately calibrate the pose relationship between the image marker and the patient in the virtual and real space. The patient model after registration is wirelessly transmitted to the HoloLens device to display the AR scene. Results: The proposed approach was used to conduct accuracy verification experiments on the phantoms and volunteers, which were compared with six advanced AR calibration methods. The proposed method obtained average fusion errors of 0.70 ± 0.16 and 0.91 ± 0.13 mm in phantom and volunteer experiments, respectively. The fusion accuracy of the proposed method is the highest among all comparison methods. A volunteer liver puncture clinical simulation experiment was also conducted to show the clinical feasibility. Conclusions: Our experiments proved the effectiveness of the proposed AR calibration method, and revealed a considerable potential for improving surgical performance.
Semi-Automatic Infrared Calibration for Augmented Reality Systems in Surgery

Hisham Iqbal,Ferdinando Rodriguez y Baena

DOI: https://doi.org/10.1109/IROS47612.2022.9982215

2024-05-03

Abstract:Augmented reality (AR) has the potential to improve the immersion and efficiency of computer-assisted orthopaedic surgery (CAOS) by allowing surgeons to maintain focus on the operating site rather than external displays in the operating theatre. Successful deployment of AR to CAOS requires a calibration that can accurately calculate the spatial relationship between real and holographic objects. Several studies attempt this calibration through manual alignment or with additional fiducial markers in the surgical scene. We propose a calibration system that offers a direct method for the calibration of AR head-mounted displays (HMDs) with CAOS systems, by using infrared-reflective marker-arrays widely used in CAOS. In our fast, user-agnostic setup, a HoloLens 2 detected the pose of marker arrays using infrared response and time-of-flight depth obtained through sensors onboard the HMD. Registration with a commercially available CAOS system was achieved when an IR marker-array was visible to both devices. Study tests found relative-tracking mean errors of 2.03 mm and 1.12° when calculating the relative pose between two static marker-arrays at short ranges. When using the calibration result to provide in-situ holographic guidance for a simulated wire-insertion task, a pre-clinical test reported mean errors of 2.07 mm and 1.54° when compared to a pre-planned trajectory.

Robotics
Real-Time Computer-Generated Integral Imaging and 3d Image Calibration for Augmented Reality Surgical Navigation

Junchen Wang,Hideyuki Suenaga,Hongen Liao,Kazuto Hoshi,Liangjing Yang,Etsuko Kobayashi,Ichiro Sakuma

DOI: https://doi.org/10.1016/j.compmedimag.2014.11.003

IF: 7.422

2015-01-01

Computerized Medical Imaging and Graphics

Abstract:Autostereoscopic 3D image overlay for augmented reality (AR) based surgical navigation has been studied and reported many times. For the purpose of surgical overlay, the 3D image is expected to have the same geometric shape as the original organ, and can be transformed to a specified location for image overlay. However, how to generate a 3D image with high geometric fidelity and quantitative evaluation of 3D image's geometric accuracy have not been addressed. This paper proposes a graphics processing unit (GPU) based computer-generated integral imaging pipeline for real-time autostereoscopic 3D display, and an automatic closed-loop 3D image calibration paradigm for displaying undistorted 3D images. Based on the proposed methods, a novel AR device for 3D image surgical overlay is presented, which mainly consists of a 3D display, an AR window, a stereo camera for 3D measurement, and a workstation for information processing. The evaluation on the 3D image rendering performance with 2560×1600 elemental image resolution shows the rendering speeds of 50–60 frames per second (fps) for surface models, and 5–8 fps for large medical volumes. The evaluation of the undistorted 3D image after the calibration yields sub-millimeter geometric accuracy. A phantom experiment simulating oral and maxillofacial surgery was also performed to evaluate the proposed AR overlay device in terms of the image registration accuracy, 3D image overlay accuracy, and the visual effects of the overlay. The experimental results show satisfactory image registration and image overlay accuracy, and confirm the system usability.
Calibration of RGBD camera and cone-beam CT for 3D intra-operative mixed reality visualization

Sing Chun Lee,Bernhard Fuerst,Javad Fotouhi,Marius Fischer,Greg Osgood,Nassir Navab

DOI: https://doi.org/10.1007/s11548-016-1396-1

2016-04-08

International Journal of Computer Assisted Radiology and Surgery

Abstract:PurposeThis work proposes a novel algorithm to register cone-beam computed tomography (CBCT) volumes and 3D optical (RGBD) camera views. The co-registered real-time RGBD camera and CBCT imaging enable a novel augmented reality solution for orthopedic surgeries, which allows arbitrary views using digitally reconstructed radiographs overlaid on the reconstructed patient’s surface without the need to move the C-arm.MethodsAn RGBD camera is rigidly mounted on the C-arm near the detector. We introduce a calibration method based on the simultaneous reconstruction of the surface and the CBCT scan of an object. The transformation between the two coordinate spaces is recovered using Fast Point Feature Histogram descriptors and the Iterative Closest Point algorithm.ResultsSeveral experiments are performed to assess the repeatability and the accuracy of this method. Target registration error is measured on multiple visual and radio-opaque landmarks to evaluate the accuracy of the registration. Mixed reality visualizations from arbitrary angles are also presented for simulated orthopedic surgeries.ConclusionTo the best of our knowledge, this is the first calibration method which uses only tomographic and RGBD reconstructions. This means that the method does not impose a particular shape of the phantom. We demonstrate a marker-less calibration of CBCT volumes and 3D depth cameras, achieving reasonable registration accuracy. This design requires a one-time factory calibration, is self-contained, and could be integrated into existing mobile C-arms to provide real-time augmented reality views from arbitrary angles.

engineering, biomedical,radiology, nuclear medicine & medical imaging,surgery
STTAR: Surgical Tool Tracking Using Off-the-Shelf Augmented Reality Head-Mounted Displays

Alejandro Martin-Gomez,Haowei Li,Tianyu Song,Sheng Yang,Guangzhi Wang,Hui Ding,Nassir Navab,Zhe Zhao,Mehran Armand

DOI: https://doi.org/10.1109/TVCG.2023.3238309

Abstract:The use of Augmented Reality (AR) for navigation purposes has shown beneficial in assisting physicians during the performance of surgical procedures. These applications commonly require knowing the pose of surgical tools and patients to provide visual information that surgeons can use during the performance of the task. Existing medical-grade tracking systems use infrared cameras placed inside the Operating Room (OR) to identify retro-reflective markers attached to objects of interest and compute their pose. Some commercially available AR Head-Mounted Displays (HMDs) use similar cameras for self-localization, hand tracking, and estimating the objects' depth. This work presents a framework that uses the built-in cameras of AR HMDs to enable accurate tracking of retro-reflective markers without the need to integrate any additional electronics into the HMD. The proposed framework can simultaneously track multiple tools without having previous knowledge of their geometry and only requires establishing a local network between the headset and a workstation. Our results show that the tracking and detection of the markers can be achieved with an accuracy of 0.09±0.06 mm on lateral translation, 0.42 ±0.32 mm on longitudinal translation and 0.80 ±0.39° for rotations around the vertical axis. Furthermore, to showcase the relevance of the proposed framework, we evaluate the system's performance in the context of surgical procedures. This use case was designed to replicate the scenarios of k-wire insertions in orthopedic procedures. For evaluation, seven surgeons were provided with visual navigation and asked to perform 24 injections using the proposed framework. A second study with ten participants served to investigate the capabilities of the framework in the context of more general scenarios. Results from these studies provided comparable accuracy to those reported in the literature for AR-based navigation procedures.
C-ARM POSE ESTIMATION AND NAVIGATION IN SURGERIES FOR AUGMENTED REALITY APPLICATION

S. Hosseinian,H. Arefi,N. Navab

DOI: https://doi.org/10.5194/isprs-archives-xlii-4-w18-497-2019

2019-10-18

Abstract:Abstract. C-arm X-ray imaging systems are widely applied in surgeries. Overlaying X-ray with optical images during the surgery has been shown to be an efficient approach. Moreover, overlaying needed data from different modalities in an augmented reality (AR) manner can improve the accuracy of surgical procedures, decrease the variability of surgical outcomes, reduce trauma to the critical structures, increase the reproducibility of surgeons’ performance, and reduce radiation exposure. C-Arm geometric calibration and recovering the C-arm pose are essential for surgical navigation and AR applications in operating rooms. Therefore, in this paper, existing researches for calibration and pose estimation of C-arm devices in surgical AR applications are evaluated from photogrammetric point of view. Then, a proposed marker-based method for C-arm pose estimation is introduced. For this purpose, a marker is designed to facilitate tracking and pose estimation in mixed reality based on golden section principle, and perspective invariants such as cross-ratios, collinearity, and intersection. Moreover, a procedure is also proposed for fast determination of these fiducial markers. The experiments show benefits of such a structure which has a limited occlusion with consistency to different conditions such as narrow field of view and at the same time, even in images with high projectivity. It also results that the distortion correction step is important and the effect of distortion of X-ray images can cause inconsistency in the perspective invariants.
Antibiotic use and the risk of flare of inflammatory bowel disease.

F. Aberra,C. Brensinger,W. Bilker,G. Lichtenstein,J. Lewis

DOI: https://doi.org/10.1016/S1542-3565(05)00020-0

IF: 13.576

2005-05-01

Clinical Gastroenterology and Hepatology

Abstract:
Spreading messages

Ching-Lueh Chang,Y. Lyuu

DOI: https://doi.org/10.1016/j.tcs.2009.03.032

IF: 1.002

2008-06-27

Theoretical Computer Science

Abstract:
Unified Calibration Technique for Augmented-Reality Ultrasound-Guided Interventions

E. Chen,Joeana Cambranis Romero,Daniel R. Allen,T. Peters

DOI: https://doi.org/10.1109/MetroXRAINE54828.2022.9967585

2022-10-26

Abstract:Accurate spatial calibration for mobile imaging modality is the essential and enabling technology for augmented-reality based surgical navigation systems. Despite years of research, spatial calibration for surgical camera and freehand ultrasound remains areas of active research. In this paper, we present a unified spatial calibration framework for ultrasound probe calibration and camera hand-eye calibration using the same mathematical principle. By treating spatial calibration as a registration problem between paired points and lines, our framework provides i) efficient solutions with guaranteed convergence properties, and ii) based on error propagation model, a set of heuristic rules for fiducial placements that leads to accurate calibration consistently. Monte Carlo simulation demonstrated that accuracy camera hand-eye calibration (≈ 5 pixel) is possible with the Microsoft HoloLens 2 using as few as 6 fiducial measurements, and accurate ultrasound probe calibration can be consistently obtained using as few as 12 fiducial measurements.

Medicine,Computer Science,Engineering
Closing the Calibration Loop: An Inside-out-tracking Paradigm for Augmented Reality in Orthopedic Surgery

Jonas Hajek,Mathias Unberath,Javad Fotouhi,Bastian Bier,Sing Chun Lee,Greg Osgood,Andreas Maier,Mehran Armand,Nassir Navab

DOI: https://doi.org/10.48550/arXiv.1803.08610

2018-03-22

Computer Vision and Pattern Recognition

Abstract:In percutaneous orthopedic interventions the surgeon attempts to reduce and fixate fractures in bony structures. The complexity of these interventions arises when the surgeon performs the challenging task of navigating surgical tools percutaneously only under the guidance of 2D interventional X-ray imaging. Moreover, the intra-operatively acquired data is only visualized indirectly on external displays. In this work, we propose a flexible Augmented Reality (AR) paradigm using optical see-through head mounted displays. The key technical contribution of this work includes the marker-less and dynamic tracking concept which closes the calibration loop between patient, C-arm and the surgeon. This calibration is enabled using Simultaneous Localization and Mapping of the environment of the operating theater. In return, the proposed solution provides in situ visualization of pre- and intra-operative 3D medical data directly at the surgical site. We demonstrate pre-clinical evaluation of a prototype system, and report errors for calibration and target registration. Finally, we demonstrate the usefulness of the proposed inside-out tracking system in achieving "bull's eye" view for C-arm-guided punctures. This AR solution provides an intuitive visualization of the anatomy and can simplify the hand-eye coordination for the orthopedic surgeon.
Seamless Augmented Reality Integration in Arthroscopy: A Pipeline for Articular Reconstruction and Guidance

Hongchao Shu,Mingxu Liu,Lalithkumar Seenivasan,Suxi Gu,Ping-Cheng Ku,Jonathan Knopf,Russell Taylor,Mathias Unberath

2024-10-01

Abstract:Arthroscopy is a minimally invasive surgical procedure used to diagnose and treat joint problems. The clinical workflow of arthroscopy typically involves inserting an arthroscope into the joint through a small incision, during which surgeons navigate and operate largely by relying on their visual assessment through the arthroscope. However, the arthroscope's restricted field of view and lack of depth perception pose challenges in navigating complex articular structures and achieving surgical precision during procedures. Aiming at enhancing intraoperative awareness, we present a robust pipeline that incorporates simultaneous localization and mapping, depth estimation, and 3D Gaussian splatting to realistically reconstruct intra-articular structures solely based on monocular arthroscope video. Extending 3D reconstruction to Augmented Reality (AR) applications, our solution offers AR assistance for articular notch measurement and annotation anchoring in a human-in-the-loop manner. Compared to traditional Structure-from-Motion and Neural Radiance Field-based methods, our pipeline achieves dense 3D reconstruction and competitive rendering fidelity with explicit 3D representation in 7 minutes on average. When evaluated on four phantom datasets, our method achieves RMSE = 2.21mm reconstruction error, PSNR = 32.86 and SSIM = 0.89 on average. Because our pipeline enables AR reconstruction and guidance directly from monocular arthroscopy without any additional data and/or hardware, our solution may hold the potential for enhancing intraoperative awareness and facilitating surgical precision in arthroscopy. Our AR measurement tool achieves accuracy within 1.59 +/- 1.81mm and the AR annotation tool achieves a mIoU of 0.721.

Computer Vision and Pattern Recognition,Machine Learning
An augmented reality and high-speed optical tracking system for laparoscopic surgery

Nati Nawawithan,Jeff Young,Patric Bettati,Armand P Rathgeb,Kelden T Pruitt,Jordan Frimpter,Henry Kim,Jonathan Yu,Davis Driver,Amanuel Shiferaw,Aditi Chaudhari,Brett A Johnson,Jeffrey Gahan,James Yu,Baowei Fei

DOI: https://doi.org/10.1117/12.3008448

Abstract:While minimally invasive laparoscopic surgery can help reduce blood loss, reduce hospital time, and shorten recovery time compared to open surgery, it has the disadvantages of limited field of view and difficulty in locating subsurface targets. Our proposed solution applies an augmented reality (AR) system to overlay pre-operative images, such as those from magnetic resonance imaging (MRI), onto the target organ in the user's real-world environment. Our system can provide critical information regarding the location of subsurface lesions to guide surgical procedures in real time. An infrared motion tracking camera system was employed to obtain real-time position data of the patient and surgical instruments. To perform hologram registration, fiducial markers were used to track and map virtual coordinates to the real world. In this study, phantom models of each organ were constructed to test the reliability and accuracy of the AR-guided laparoscopic system. Root mean square error (RMSE) was used to evaluate the targeting accuracy of the laparoscopic interventional procedure. Our results demonstrated a registration error of 2.42 ± 0.79 mm and a procedural targeting error of 4.17 ± 1.63 mm using our AR-guided laparoscopic system that will be further refined for potential clinical procedures.
Deconstructing the molecular mechanisms of cell cycle control in a mouse adrenocortical cell line: Roles of ACTH

K. Rocha,F. L. Forti,A. Lepique,H. Armelin

DOI: https://doi.org/10.1002/jemt.10336

2003-06-15

Abstract:This is a progress report of an attempt to deconstruct the signaling network underlying cell cycle control in the mouse Y1 adrenocortical cell line, aiming to uncover ACTH growth regulatory pathways. Y1 adrenocortical tumor cells possess amplified and overexpressed c‐Ki‐ras proto‐oncogene. Despite this oncogenic lesion, Y1 cells retain tight regulatory mechanisms of cell cycle control typified by the sequential events comprising the mitogenic response triggered by FGF2 in G0/G1‐arrested Y1 cells: 1) activation of ERK1/2 and PI3K, by 5 minutes; 2) induction of c‐Fos and c‐Myc proteins by 2 hours; 3) induction of cyclin D1 protein by 5 hours; 4) phosphorylation of Rb protein between 6 and 8 hours; 5) onset of DNA synthesis by 8–9 hours. In this cell line, ACTH‐receptor (ACTH‐R) activates contradictory pathways of growth regulation. First, ACTH coordinately induces fos and jun gene families via activation of both ERK1/2 and cAMP/PKA pathways, resembling a mitogen. Second, ACTH‐R triggers cAMP/PKA‐mediated antimitogenic mechanisms comprised of Akt/PKB dephosphorylation/deactivation, c‐Myc protein degradation, and p27Kip1 protein induction. Induction of cyclin D1 depends on activation of both ERK1/2 and PI3K, but is not affected by ACTH action. As a consequence, ACTH antagonizes FGF2 mitogenic activity but ectopic expression of the c‐Myc protein (via MycER fusion protein) is sufficient to abrogate this ACTH antagonistic effect over FGF2 mitogenic activity. Ectopic expression of both c‐Myc and cyclin D1 is not sufficient to drive G0/G1‐arrested Y1 cells into S phase, but when the sustained expression of these two proteins is complemented by ACTH treatment it promotes G1 phase progression and DNA synthesis initiation. In conclusion, ACTH‐receptor lacks signaling potential sufficient to initiate a mitogenic response in Y1 adrenocortical cells and, therefore, cannot substitute for bona fide mitogens like FGF2. Microsc. Res. Tech. 61:268–274, 2003. © 2003 Wiley‐Liss, Inc.
Gap formation in the density of states for the Holstein model.

Vekic,White

DOI: https://doi.org/10.1103/PHYSREVB.48.7643

1993-09-01

Abstract:We study the electron and phonon spectral-weight functions and densities of states for the Holstein model obtained by analytically continuing numerical single-particle Green's functions. At half-filling we observe the appearance of a charge-density-wave gap in the electron spectral-weight function and a softening and broadening of the phonon spectrum in the density of states as a function of the inverse temperature. Away from half-filling we have a transition to a superconducting phase indicated by the formation of a gap in the electron density of states. Our results are consistent with previously studied correlation functions, susceptibilities, and self-energies obtained from quantum Monte Carlo calculations and diagrammatic techniques.
3D Surgical Overlay with Markerless Image Registration Using a Single Camera.

Junchen Wang,Hideyuki Suenaga,Liangjing Yang,Hongen Liao,Takehiro Ando,Etsuko Kobayashi,Ichiro Sakuma

DOI: https://doi.org/10.1007/978-3-319-24601-7_13

2015-01-01

Abstract:Minimum invasive surgery can benefit from surgical visualization, which is achieved by either virtual reality or augmented reality. We previously proposed an integrated 3D image overlay based surgical visualization solution including 3D image rendering, distortion correction, and spatial projection. For correct spatial projection of the 3D image, image registration is necessary. In this paper we present a 3D image overlay based augmented reality surgical navigation system with markerless image registration using a single camera. The innovation compared with our previous work lies in the single camera based image registration method for 3D image overlay. The 3D mesh model of patient’s teeth which is created from the preoperative CT data is matched with the intraoperative image captured by a single optical camera to determine the six-degree-of-freedom pose of the model with respect to the camera. The obtained pose is used to superimpose the 3D image of critical hidden tissues on patient’s body directly via a translucent mirror for surgical visualization. The image registration performs automatically within approximate 0.2 s, which enables real-time update to tackle patient’s movement. Experimental results show that the registration accuracy is about 1 mm and confirm the feasibility of the 3D surgical overlay system.
Robust hand tracking for surgical telestration

Lucas-Raphael Müller,Jens Petersen,Amine Yamlahi,Philipp Wise,Tim J Adler,Alexander Seitel,Karl-Friedrich Kowalewski,Beat Müller,Hannes Kenngott,Felix Nickel,Lena Maier-Hein

DOI: https://doi.org/10.1007/s11548-022-02637-9

Abstract:Purpose: As human failure has been shown to be one primary cause for post-operative death, surgical training is of the utmost socioeconomic importance. In this context, the concept of surgical telestration has been introduced to enable experienced surgeons to efficiently and effectively mentor trainees in an intuitive way. While previous approaches to telestration have concentrated on overlaying drawings on surgical videos, we explore the augmented reality (AR) visualization of surgical hands to imitate the direct interaction with the situs. Methods: We present a real-time hand tracking pipeline specifically designed for the application of surgical telestration. It comprises three modules, dedicated to (1) the coarse localization of the expert's hand and the subsequent (2) segmentation of the hand for AR visualization in the field of view of the trainee and (3) regression of keypoints making up the hand's skeleton. The semantic representation is obtained to offer the ability for structured reporting of the motions performed as part of the teaching. Results: According to a comprehensive validation based on a large data set comprising more than 14,000 annotated images with varying application-relevant conditions, our algorithm enables real-time hand tracking and is sufficiently accurate for the task of surgical telestration. In a retrospective validation study, a mean detection accuracy of 98%, a mean keypoint regression accuracy of 10.0 px and a mean Dice Similarity Coefficient of 0.95 were achieved. In a prospective validation study, it showed uncompromised performance when the sensor, operator or gesture varied. Conclusion: Due to its high accuracy and fast inference time, our neural network-based approach to hand tracking is well suited for an AR approach to surgical telestration. Future work should be directed to evaluating the clinical value of the approach.
Real-time Geometry-Aware Augmented Reality in Minimally Invasive Surgery

Long Chen,Wen Tang,Nigel W. John

DOI: https://doi.org/10.48550/arXiv.1708.01234

2017-08-04

Abstract:The potential of Augmented Reality (AR) technology to assist minimally invasive surgeries (MIS) lies in its computational performance and accuracy in dealing with challenging MIS scenes. Even with the latest hardware and software technologies, achieving both real-time and accurate augmented information overlay in MIS is still a formidable task. In this paper, we present a novel real-time AR framework for MIS that achieves interactive geometric aware augmented reality in endoscopic surgery with stereo views. Our framework tracks the movement of the endoscopic camera and simultaneously reconstructs a dense geometric mesh of the MIS scene. The movement of the camera is predicted by minimising the re-projection error to achieve a fast tracking performance, while the 3D mesh is incrementally built by a dense zero mean normalised cross correlation stereo matching method to improve the accuracy of the surface reconstruction. Our proposed system does not require any prior template or pre-operative scan and can infer the geometric information intra-operatively in real-time. With the geometric information available, our proposed AR framework is able to interactively add annotations, localisation of tumours and vessels, and measurement labelling with greater precision and accuracy compared with the state of the art approaches.

Computer Vision and Pattern Recognition
Augmented Reality Navigation with Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery

Junchen Wang,Hideyuki Suenaga,Kazuto Hoshi,Liangjing Yang,Etsuko Kobayashi,Ichiro Sakuma,Hongen Liao

DOI: https://doi.org/10.1109/tbme.2014.2301191

2014-01-01

Abstract:Computer-assisted oral and maxillofacial surgery (OMS) has been rapidly evolving since the last decade. State-of-the-art surgical navigation in OMS still suffers from bulky tracking sensors, troublesome image registration procedures, patient movement, loss of depth perception in visual guidance, and low navigation accuracy. We present an augmented reality navigation system with automatic marker-free image registration using 3-D image overlay and stereo tracking for dental surgery. A customized stereo camera is designed to track both the patient and instrument. Image registration is performed by patient tracking and real-time 3-D contour matching, without requiring any fiducial and reference markers. Real-time autostereoscopic 3-D imaging is implemented with the help of a consumer-level graphics processing unit. The resulting 3-D image of the patient's anatomy is overlaid on the surgical site by a half-silvered mirror using image registration and IP-camera registration to guide the surgeon by exposing hidden critical structures. The 3-D image of the surgical instrument is also overlaid over the real one for an augmented display. The 3-D images present both stereo and motion parallax from which depth perception can be obtained. Experiments were performed to evaluate various aspects of the system; the overall image overlay error of the proposed system was 0.71 mm.
Augmented Reality-based Feedback for Technician-in-the-loop C-arm Repositioning

Mathias Unberath,Javad Fotouhi,Jonas Hajek,Andreas Maier,Greg Osgood,Russell Taylor,Mehran Armand,Nassir Navab

DOI: https://doi.org/10.48550/arXiv.1806.08814

2018-06-23

Abstract:Interventional C-arm imaging is crucial to percutaneous orthopedic procedures as it enables the surgeon to monitor the progress of surgery on the anatomy level. Minimally invasive interventions require repeated acquisition of X-ray images from different anatomical views to verify tool placement. Achieving and reproducing these views often comes at the cost of increased surgical time and radiation dose to both patient and staff. This work proposes a marker-free "technician-in-the-loop" Augmented Reality (AR) solution for C-arm repositioning. The X-ray technician operating the C-arm interventionally is equipped with a head-mounted display capable of recording desired C-arm poses in 3D via an integrated infrared sensor. For C-arm repositioning to a particular target view, the recorded C-arm pose is restored as a virtual object and visualized in an AR environment, serving as a perceptual reference for the technician. We conduct experiments in a setting simulating orthopedic trauma surgery. Our proof-of-principle findings indicate that the proposed system can decrease the 2.76 X-ray images required per desired view down to zero, suggesting substantial reductions of radiation dose during C-arm repositioning. The proposed AR solution is a first step towards facilitating communication between the surgeon and the surgical staff, improving the quality of surgical image acquisition, and enabling context-aware guidance for surgery rooms of the future. The concept of technician-in-the-loop design will become relevant to various interventions considering the expected advancements of sensing and wearable computing in the near future.

Computer Vision and Pattern Recognition

Cognitive rehearsal in the treatment of obesity: a comparison against cue avoidance and social pressure.

Towards Robotic Eye Surgery: Marker-free, Online Hand-eye Calibration using Optical Coherence Tomography Images

Augmented reality calibration using feature triangulation iteration-based registration for surgical navigation

Semi-Automatic Infrared Calibration for Augmented Reality Systems in Surgery

Real-Time Computer-Generated Integral Imaging and 3d Image Calibration for Augmented Reality Surgical Navigation

Calibration of RGBD camera and cone-beam CT for 3D intra-operative mixed reality visualization

STTAR: Surgical Tool Tracking Using Off-the-Shelf Augmented Reality Head-Mounted Displays

C-ARM POSE ESTIMATION AND NAVIGATION IN SURGERIES FOR AUGMENTED REALITY APPLICATION

Antibiotic use and the risk of flare of inflammatory bowel disease.

Spreading messages

Unified Calibration Technique for Augmented-Reality Ultrasound-Guided Interventions

Closing the Calibration Loop: An Inside-out-tracking Paradigm for Augmented Reality in Orthopedic Surgery

Seamless Augmented Reality Integration in Arthroscopy: A Pipeline for Articular Reconstruction and Guidance

An augmented reality and high-speed optical tracking system for laparoscopic surgery

Deconstructing the molecular mechanisms of cell cycle control in a mouse adrenocortical cell line: Roles of ACTH

Gap formation in the density of states for the Holstein model.

3D Surgical Overlay with Markerless Image Registration Using a Single Camera.

Robust hand tracking for surgical telestration

Real-time Geometry-Aware Augmented Reality in Minimally Invasive Surgery

Augmented Reality Navigation with Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery

Augmented Reality-based Feedback for Technician-in-the-loop C-arm Repositioning