Live image-based neurosurgical guidance and roadmap generation using unsupervised embedding

Gary Sarwin,Alessandro Carretta,Victor Staartjes,Matteo Zoli,Diego Mazzatenta,Luca Regli,Carlo Serra,Ender Konukoglu

DOI: https://doi.org/10.1007/978-3-031-34048-2_9

2023-03-31

Abstract:Advanced minimally invasive neurosurgery navigation relies mainly on Magnetic Resonance Imaging (MRI) guidance. MRI guidance, however, only provides pre-operative information in the majority of the cases. Once the surgery begins, the value of this guidance diminishes to some extent because of the anatomical changes due to surgery. Guidance with live image feedback coming directly from the surgical device, e.g., endoscope, can complement MRI-based navigation or be an alternative if MRI guidance is not feasible. With this motivation, we present a method for live image-only guidance leveraging a large data set of annotated neurosurgical <a class="link-external link-http" href="http://videos.First" rel="external noopener nofollow">this http URL</a>, we report the performance of a deep learning-based object detection method, YOLO, on detecting anatomical structures in neurosurgical images. Second, we present a method for generating neurosurgical roadmaps using unsupervised embedding without assuming exact anatomical matches between patients, presence of an extensive anatomical atlas, or the need for simultaneous localization and mapping. A generated roadmap encodes the common anatomical paths taken in surgeries in the training set. At inference, the roadmap can be used to map a surgeon's current location using live image feedback on the path to provide guidance by being able to predict which structures should appear going forward or backward, much like a mapping application. Even though the embedding is not supervised by position information, we show that it is correlated to the location inside the brain and on the surgical path. We trained and evaluated the proposed method with a data set of 166 transsphenoidal adenomectomy procedures.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to provide real - time anatomical structure navigation in neurosurgery. Specifically, the existing neurosurgical navigation mainly relies on preoperative magnetic resonance imaging (MRI). However, once the operation starts, due to the anatomical structure changes caused by the operation, the navigation value of this preoperative image will be significantly reduced. Therefore, this paper proposes a method based on real - time image feedback, using unsupervised embedding techniques to generate neurosurgical path maps to supplement or replace MRI navigation. This method does not require the assumption of precise anatomical matching between patients, nor does it require widely used anatomical atlases, or simultaneous localization and mapping. Through this method, it is possible to predict which anatomical structures should appear during the forward or backward movement of the surgical path, thus providing guidance for surgeons.

The key points of the paper include:
- Using deep - learning methods (such as YOLO) to detect anatomical structures in neurosurgical images.
- Using unsupervised embedding techniques to generate neurosurgical path maps, which encode common surgical paths in the training set.
- Mapping the current position of the surgeon through real - time image feedback and being able to predict subsequent or previous anatomical structures.

The innovation of this method lies in its ability to provide dynamic, real - time - image - based navigation during the operation without the need for additional expensive equipment or complex preoperative preparations. This helps to improve the safety and efficiency of the operation.