Design of a Surgical Human-Machine Natural Interaction Assistance System based on Holographic Reality.

Jianren Chen,Tianzuo Liu,Xiaoxue Sun,Hongpeng Wang,Jianda Han
DOI: https://doi.org/10.1109/ROBIO58561.2023.10354935
2023-01-01
Abstract:Ideally, intelligent surgery should allow surgeons to achieve insight, prediction, and assessment of the full range and course of diseased tissue without causing perceptual and interactive inconvenience. However, in the clinical setting, surgeons can only achieve judgment of the condition by continuously exposing patient tissues and organs through dissection, which is risky. To address these issues, this paper proposes a surgical human-machine natural interaction assistance system based on Holographic Reality. First, we annotate and segment the patient’s medical images and plan the surgical path to generate a knowledge-based 3D digital human model containing information about the tissue structure of the lesion and the access to the surgical instruments. Secondly, we introduce techniques such as motion recognition and force feedback into the virtual reality environment for holographic display and somatosensory interaction, achieving gesture interaction and virtual surgical instrument manipulation of the digital manikin. This system uses medical image processing and holographic reality technology to help surgeons perform preoperative surgical planning and intraoperative real-time information guidance to obtain accurate and safe surgical results.
What problem does this paper attempt to address?