Gaze-based Learning from Demonstration In Surgical Robotics

A.E. Abdelaal,S.N. Zaman,P.Y Chen,T. Suzuki,J. Ingleton
2023-11-01
Abstract:Surgical robotics is a rising field in medical technology and advanced robotics. Robot assisted surgery, or robotic surgery, allows surgeons to perform complicated surgical tasks with more precision, automation, and flexibility than is possible for traditional surgical approaches. The main type of robot assisted surgery is minimally invasive surgery, which could be automated and result in a faster healing time for the patient. The surgical robot we are particularly interested in is the da Vinci surgical system, which is developed and manufactured by Intuitive Surgical. In the current iteration of the system, the endoscopic camera arm on the da Vinci robot has to be manually controlled and calibrated by the surgeon during a surgical task, which interrupts the flow of the operation. The main goal of this capstone project is to automate the motion of the camera arm using a probabilistic model based on surgeon eye gaze data and da Vinci robot kinematic data.
Robotics
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to automate the control of the position and angle of the endoscope camera in surgical robots, so as to reduce the cognitive burden on surgeons during long - time surgeries. The current da Vinci surgical system requires surgeons to switch the control of the tool arm and the camera arm via foot pedals, which will interrupt the surgical process and increase the cognitive load of surgeons. The goal of the paper is to develop an automatic camera control system based on surgeons' eye - movement data and robotic kinematics data, enabling surgeons to focus on controlling the tool arm while the camera arm can autonomously adjust to the appropriate position and orientation according to the surgical task. Specifically, the paper solves this problem through the following methods: 1. **Data collection**: Use the da Vinci Research Kit (dVRK) and a custom - made eye - tracking device to collect robotic kinematics data and surgeons' eye - movement data. 2. **Model selection and training**: Adopt the Gaussian Mixture Model (GMM) as a machine - learning model. Through the Learning from Demonstration (LfD) method, use surgeons' eye - movement data and robotic kinematics data to train the model and predict the optimal camera position and angle. 3. **Model verification**: Test the performance of the model through simulation and actual da Vinci robots to ensure that the camera position and angle output by the model are reachable and effective. 4. **Expanding input parameters**: Besides the basic robotic kinematics data, also attempt to add the pose of the robotic arm (represented by quaternions) and surgeons' eye - movement data to improve the generalization ability and accuracy of the model. The main contribution of the paper lies in proposing an automatic camera control method that combines eye - movement data and robotic kinematics data, aiming to improve the fluency and continuity of the surgical process and reduce the operational burden on surgeons.