Automated Key Action Detection for Closed Reduction of Pelvic Fractures by Expert Surgeons in Robot-Assisted Surgery

Ming-Zhang Pan,Ya-Wen Deng,Zhen Li,Yuan Chen,Xiao-Lan Liao,Gui-Bin Bian

DOI: https://doi.org/10.1109/iros55552.2023.10342019

2023-01-01

Abstract:Pelvic fractures are one of the most serious traumas in orthopedics, and the technical proficiency and expertise of the surgical team strongly influence the quality of reduction results. With the advancement of information technology and robotics, robot-assisted pelvic fracture reduction surgery is expected to reduce the impact caused by inexperienced doctors and improve the accuracy and stability of pelvic reduction. However, this requires the robot to detect key surgeon actions from time-series data, enabling the robot to independently perceive the surgical status, predict the surgeon's intentions, assess the demonstrated level of professional competence, and assess the progress of the surgery. Therefore, a multi-task deep learning neural network architecture is proposed, which incorporates Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) along with tri-modality fusion and feature extraction techniques. The proposed framework aims to achieve key action detection in closed reduction operations for pelvic fractures. Subsequently, a trimodal fine-grained dataset was constructed, wherein 29, 32, and 14 labels were marked on flexion, position, and pressure data for 14 key closed reduction actions. The experimental results show that the correct detection rate of closed reduction actions is 92.3 %, significantly higher than the commonly used recognition algorithms. This work provides a method for the robot to learn the surgeon's professional knowledge, provides the basis for the operation's motion perception, and contributes to the autonomy of the robot-assisted closed reduction surgery of pelvic fractures.