Developments of Multimodal Image-Guided Surgical Navigation
Yang Jian,Wang Yuanyuan,Ai Danni,Song Hong,Fan Jingfan,Fu Tianyu,Xiao Deqiang,Shao Long,Gu Ying,Wang Yongtian
DOI: https://doi.org/10.3788/aos230742
2023-01-01
Abstract:Significance Surgical navigation combines organ segmentation modeling, surgical planning, pose calibration and tracking, multimodal image registration, and fusion display technologies to enable surgeons to precisely locate lesions and surgical tools and to observe internal tissue through the tissue surface, which can significantly improve surgical safety and time efficiency. Conventional surgery usually uses two-dimensional (2D) unimodal images such as ultrasound, endoscopy, or X-ray images to guide the surgical process. However, unimodal images lack three-dimensional (3D) information and depend heavily on the surgeon's experience. In contrast, multimodal image-guided surgical navigation provides real-time instrument positions, as well as structural and functional information of the lesion in 3D space, helping the surgeons to effectively protect important tissue, vessels, and organs around the lesion, avoiding unnecessary damage, and reducing the probability of surgical complications, which has become an important tool for a variety of clinical surgical procedures. Progress The surgical navigation system mainly consists of imaging devices, tracking and positioning core devices, end-effectors, surgical tools, and other hardware, and it combines modern imaging technology, stereotactic technology, computer technology, and artificial intelligence technology to enable patients to obtain safe, precise, and minimally invasive surgical treatment. The surgical navigation system involves core theories and methods in various aspects such as multimodal image segmentation and tissue modeling, surgical planning, pose calibration and tracking, multimodal image registration, and image fusion. The hardware components of multimodal image-guided surgical navigation mainly include intraoperative imaging devices, such as X-ray, ultrasound, and endoscopic imaging systems, tracking and positioning core devices, such as optical and electromagnetic lasers, structured light positioning systems, and navigation actuation components, such as robotic arms and guidewire. The key technologies for multimodal image-guided surgical navigation include multimodal image segmentation and tissue modeling, surgical protocol decision making, surgical spatial calibration and tracking, multimodal image registration, and multi-source information fusion display. The segmentation and modeling technologies based on preoperative multimodal medical images can depict the spatial structure and position information of target tissue and organs, providing an important data base for preoperative surgical planning and intraoperative real-time guidance. The surgical plan decision is used to guide intraoperative surgical operations, and the surgical plan can be formulated by the relationship of the 3D model positions of tissue, organs, and lesions. Preoperative planning for different surgical procedures has a large variability and can be divided into two categories: surgical path planning and surgical scheme planning. The surgical navigation system is based on core tracking and positioning devices to track the real-time position of intraoperative surgical instruments and obtain the relative position relationship of the preoperative reconstructed model, intraoperative patients, and surgical instruments. Multimodal image registration aims to seek to coordinate transformation among multimodal medical images to make these images aligned and unified in the spatial coordinate system, which helps to obtain complementary tissue structure or functional information from different modalities. The fusion and display of multi-source information aim to integrate different images, tissue models, surgical protocols, tracking postures, and other information on the same coordinate system for 2D or 3D display, which overcomes the limitations of a single source in the information presentation and contributes to improving the precision of clinical diagnosis and treatment. At present, multimodal image-guided surgical navigation has become a powerful tool for precise treatment in clinical departments such as neurosurgery, craniomaxillofacial, orthopedics, percutaneous puncture, and vascular intervention, and it has important application prospects. Conclusions and Prospects Multimodal image-guided surgical navigation provides structural and functional information of the lesion in 3D space by fusing multimodal images such as X-ray, endoscopy, ultrasound, and fluorescence, as well as integrating technologies such as multimodal image segmentation and tissue modeling, surgical planning and decision making, pose calibration and tracking, and multimodal image registration and fusion to improve the surgeon's visual perception and spatial recognition of important tissue such as blood vessels and nerves. This technique significantly improves the safety of surgery, shortens surgery time, and increases surgery efficiency. Multimodal image-guided surgical navigation has been widely used in minimally invasive surgeries such as neurosurgery, craniomaxillofacial orthopedics, orthopedics, puncture, and vascular interventions, which can assist surgeons to achieve precise treatment with less trauma, higher efficacy, and faster recovery and reduce complications of surgery, which is a major frontier hotspot in the international precision treatment. Achieving higher precision, higher intelligence, lighter weight, and lower cost of navigation devices are the main development directions of surgical navigation. How to further break the difficulties of image registration, deformation compensation, motion compensation, and soft tissue tension perception techniques, how to develop a high-precision non-rigid registration algorithm for flexible tissue deformation, and how to achieve dynamic visualization of intraoperative navigation information and decision making of surgical plans are essential to further improve the real-time, portability, accuracy, and intelligence of current surgical navigation systems. In the future, as the application scenarios of multimodal surgical navigation systems in the clinic continue to expand, the clinical application value of surgical navigation technology will become prominent, which will provide reliable guidance and assistance to more surgeons, and it is of great significance in improving the level of minimally invasive surgical treatment in China.