Abstract:Abstract Background Safe and accurate execution of surgeries to date mainly rely on preoperative plans generated based on preoperative imaging. Frequent intraoperative interaction with such patient images during the intervention is needed, which is currently a cumbersome process given that such images are generally displayed on peripheral two-dimensional (2D) monitors and controlled through interface devices that are outside the sterile filed. This study proposes a new medical image control concept based on a Brain Computer Interface (BCI) that allows for hands-free and direct image manipulation without relying on gesture recognition methods or voice commands. Method A software environment was designed for displaying three-dimensional (3D) patient images onto external monitors, with the functionality of hands-free image manipulation based on the user’s brain signals detected by the BCI device (i.e., visually evoked signals). In a user study, ten orthopedic surgeons completed a series of standardized image manipulation tasks to navigate and locate predefined 3D points in a Computer Tomography (CT) image using the developed interface. Accuracy was assessed as the mean error between the predefined locations (ground truth) and the navigated locations by the surgeons. All surgeons rated the performance and potential intraoperative usability in a standardized survey using a five-point Likert scale (1 = strongly disagree to 5 = strongly agree). Results When using the developed interface, the mean image control error was 15.51 mm (SD: 9.57). The user's acceptance was rated with a Likert score of 4.07 (SD: 0.96) while the overall impressions of the interface was rated as 3.77 (SD: 1.02) by the users. We observed a significant correlation between the users' overall impression and the calibration score they achieved. Conclusions The use of the developed BCI, that allowed for a purely brain-guided medical image control, yielded promising results, and showed its potential for future intraoperative applications. The major limitation to overcome was noted as the interaction delay.

An Analysis Framework for Near InfraRed Spectroscopy Based Brain-Computer Interface and Prospective Application to Robotic Surgery

Brain Activity Analysis for the Configurational Determination of Surgical Manipulation and Endoscope

Automated Task Load Detection with Electroencephalography: Towards Passive Brain-Computer Interfacing in Robotic Surgery.

Introducing a brain-computer interface to facilitate intraoperative medical imaging control – a feasibility study

A Semi-Autonomous Stereotactic Brain Biopsy Robotic System With Enhanced Surgical Safety and Surgeon-Robot Collaboration

Cognitive Burden Estimation for Visuomotor Learning with Fnirs

Development of an AI-driven system for neurosurgery with a usability study: a step towards minimal invasive robotics

Multimodal data fusion framework enhanced robot-assisted minimally invasive surgery

Gaze-contingent perceptually enabled interactions in the operating theatre

Physiological Metrics of Surgical Difficulty and Multi-Task Requirement during Robotic Surgery Skills

Sensory Glove-Based Surgical Robot User Interface

Challenges and Countermeasures of Brain Computer Interface Technology Based on Functional Near Infrared Spectroscopy for Aerospace Applications

Quantification of Robotic Surgeries with Vision-Based Deep Learning

Hybrid Robot-Assisted Frameworks for Endomicroscopy Scanning in Retinal Surgeries

Multimodal Physiological Signals for Workload Prediction in Robot-assisted Surgery

ST-ITEF: Spatio-Temporal Intraoperative Task Estimating Framework to recognize surgical phase and predict instrument path based on multi-object tracking in keratoplasty

Combination of Augmented Reality Based Brain-Computer Interface and EyeTracking for Control of a Multi-Robot System

Functional Near Infrared Spectroscopy in Novice and Expert Surgeons - A Manifold Embedding Approach

Toward Intraoperative Visual Intelligence: Real-Time Surgical Instrument Segmentation for Enhanced Surgical Monitoring

Cognitive burden estimation for visuomotor learning with fNIRS.

Vision-Based Framework of Single Master Dual Slave Semi-Autonomous Surgical Robot System