Abstract:Purpose: Minimally invasive surgery (MIS) has become the standard for many surgical procedures as it minimizes trauma, reduces infection rates and shortens hospitalization. However, the manipulation of objects in the surgical workspace can be difficult due to the unintuitive handling of instruments and limited range of motion. Apart from the advantages of robot-assisted systems such as augmented view or improved dexterity, both robotic and MIS techniques introduce drawbacks such as limited haptic perception and their major reliance on visual perception. Methods: In order to address the above-mentioned limitations, a perception study was conducted to investigate whether the transmission of intra-abdominal acoustic signals can potentially improve the perception during MIS. To investigate whether these acoustic signals can be used as a basis for further automated analysis, a large audio data set capturing the application of electrosurgery on different types of porcine tissue was acquired. A sliding window technique was applied to compute log-mel-spectrograms, which were fed to a pre-trained convolutional neural network for feature extraction. A fully connected layer was trained on the intermediate feature representation to classify instrument-tissue interaction. Results: The perception study revealed that acoustic feedback has potential to improve the perception during MIS and to serve as a basis for further automated analysis. The proposed classification pipeline yielded excellent performance for four types of instrument-tissue interaction (muscle, fascia, liver and fatty tissue) and achieved top-1 accuracies of up to 89.9%. Moreover, our model is able to distinguish electrosurgical operation modes with an overall classification accuracy of 86.40%. Conclusion: Our proof-of-principle indicates great application potential for guidance systems in MIS, such as controlled tissue resection. Supported by a pilot perception study with surgeons, we believe that utilizing audio signals as an additional information channel has great potential to improve the surgical performance and to partly compensate the loss of haptic feedback.

Audio-based event detection in the operating room

Acoustic signal analysis of instrument-tissue interaction for minimally invasive interventions

The sound of surgery-development of an acoustic trocar system enabling laparoscopic sound analysis

Automating Feedback Analysis in Surgical Training: Detection, Categorization, and Assessment

Learning-based Surgical Workflow Detection from Intra-Operative Signals

Estimation and Monitoring of Operating Room Utilization by a Distributed Streaming and Analytics Architecture Deployed at Heidelberg University Hospital's Medical Data Integration Center

Intra-operative surgical instrument usage detection on a multi-sensor table

State-of-the-art of situation recognition systems for intraoperative procedures

Audio Surveillance: a Systematic Review

Anomalous sound event detection: A survey of machine learning based methods and applications

Real-world acoustic event detection

A Human-in-the-Loop Method for Annotation of Events in Biomedical Signals

Speech-Based Surgical Phase Recognition for Non-Intrusive Surgical Skills’ Assessment in Educational Contexts

Sound perception in realistic surgery scenarios: Towards EEG-based auditory work strain measures for medical personnel

Simulating environmental and psychological acoustic factors of the operating room

Using mobile EEG to study auditory work strain during simulated surgical procedures

A platform for cognitive monitoring of neurosurgical patients during hospitalization

Machine listening in a neonatal intensive care unit

Two-stage visual speech recognition for intensive care patients

Real-time image-based instrument classification for laparoscopic surgery

Auditory display as feedback for a novel eye-tracking system for sterile operating room interaction