Endoscope FOV Autonomous Tracking Method for Robot-Assisted Surgery Considering Pose Control, Hand–Eye Coordination, and Image Definition
Jianqing Peng,Chi Zhang,Liang Kang,Guodong Feng
DOI: https://doi.org/10.1109/tim.2022.3204086
IF: 5.6
2022-10-01
IEEE Transactions on Instrumentation and Measurement
Abstract:In laparoscopic surgeries, intelligent-assisted endoscope-holding robots (AEHRs) with traditional servo control have limited performance due to the lack of endoscopic attitude control, hand–eye coordination, and image definition. This article proposes an endoscope field-of-view (FOV) autonomous tracking method for robot-assisted surgery (RAS) considering pose (i.e., position and attitude) control, hand–eye coordination, and image definition, and it can realize that the AEHR system can safely, stably, and quickly track the movement of surgical instruments. First, the general framework of the intelligent tracking system with robot-assisted endoscope holding is constructed according to the clinical needs and shortcomings of traditional research. Second, the kinematic model and remote center of motion (RCM) constraint equations of AEHRs are established. Then, surgical instrument tips are segmented and localized using the deep learning method (DLM), and further, a surgical FOV intelligent adjustment control model with multiple indexes is developed. Finally, an experimental system based on the hybrid of AEHRs and speech interaction is built. The simulation and experimental results demonstrate that the proposed method can achieve fast, accurate, and effective instrument tips tracking (ITT), reduce hand–eye incoordination, and improve the safety and stability of FOV intelligent adjustment.
engineering, electrical & electronic,instruments & instrumentation