Dynamic Virtual Fixture Generation Based on Intra-Operative 3D Image Feedback in Robot-Assisted Minimally Invasive Thoracic Surgery

Yunze Shi,Peizhang Zhu,Tengyue Wang,Haonan Mai,Xiyang Yeh,Liangjing Yang,Jingfan Wang

DOI: https://doi.org/10.3390/s24020492

IF: 3.9

2024-01-13

Sensors

Abstract:This paper proposes a method for generating dynamic virtual fixtures with real-time 3D image feedback to facilitate human–robot collaboration in medical robotics. Seamless shared control in a dynamic environment, like that of a surgical field, remains challenging despite extensive research on collaborative control and planning. To address this problem, our method dynamically creates virtual fixtures to guide the manipulation of a trocar-placing robot arm using the force field generated by point cloud data from an RGB-D camera. Additionally, the "view scope" concept selectively determines the region for computational points, thereby reducing computational load. In a phantom experiment for robot-assisted port incision in minimally invasive thoracic surgery, our method demonstrates substantially improved accuracy for port placement, reducing error and completion time by 50% (p=1.06×10−2) and 35% (p=3.23×10−2), respectively. These results suggest that our proposed approach is promising in improving surgical human–robot collaboration.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to generate dynamic virtual fixtures through real - time 3D image feedback in robot - assisted minimally invasive thoracic surgery to improve the precision and efficiency of human - robot collaboration. Specifically, the paper proposes methods for the following problems:

1. **Seamless shared control in a dynamic environment**: In the surgical scenario, the environment changes dynamically, which makes seamless shared control between humans and robots a major challenge. Although existing research has made progress in collaborative control and planning, it is still difficult to achieve seamless shared control in a dynamic environment.

2. **Limitations of manual operations**: During the surgical process, providing only visual feedback is not sufficient for tasks that require "hand - feel" (such as drilling and puncturing). Therefore, in addition to visual feedback, force feedback also needs to be provided to guide surgical operations.

3. **Dynamic generation of virtual fixtures**: Traditional virtual fixture generation methods usually rely on predefined geometric shapes or anatomical structures, and these methods are cumbersome and impractical in dynamic and complex surgical environments. The paper proposes a method for generating dynamic virtual fixtures based on point - cloud data, which can adapt to changes in the surgical environment in real - time.

4. **Reducing surgical errors and time**: By generating dynamic virtual fixtures, the paper aims to improve the accuracy of port placement during surgery, reduce surgical errors and completion time.

To verify the effectiveness of the proposed method, the paper conducted a simulation experiment. The results show that this method can significantly improve the accuracy of port placement, reducing errors by 50% and completion time by 35% respectively. These results indicate that this method has great potential in improving human - robot collaboration during surgery.