Abstract:INTRODUCTION:In the modern day operating room, the surgeon performs surgeries with the support of different medical systems that showcase patient information, physiological data, and medical images. It is generally accepted that numerous interactions must be performed by the surgical team to control the corresponding medical system to retrieve the desired information. Joysticks and physical keys are still present in the operating room due to the disadvantages of mouses, and surgeons often communicate instructions to the surgical team when requiring information from a specific medical system. In this paper, a novel user interface is developed that allows the surgeon to personally perform touchless interaction with the various medical systems, switch effortlessly among them, all of this without modifying the systems' software and hardware.METHODS:To achieve this, a wearable RGB-D sensor is mounted on the surgeon's head for inside-out tracking of his/her finger with any of the medical systems' displays. Android devices with a special application are connected to the computers on which the medical systems are running, simulating a normal USB mouse and keyboard. When the surgeon performs interaction using pointing gestures, the desired cursor position in the targeted medical system display, and gestures, are transformed into general events and then sent to the corresponding Android device. Finally, the application running on the Android devices generates the corresponding mouse or keyboard events according to the targeted medical system.RESULTS AND CONCLUSION:To simulate an operating room setting, our unique user interface was tested by seven medical participants who performed several interactions with the visualization of CT, MRI, and fluoroscopy images at varying distances from them. Results from the system usability scale and NASA-TLX workload index indicated a strong acceptance of our proposed user interface.

Leap Motion Gesture Control With Carestream Software in the Operating Room to Control Imaging

The Virtual Surgery of Extraocular Muscles Based on Gesture Interaction

Device- and System-Independent Personal Touchless User Interface for Operating Rooms

Application of KinectTM and wireless technology for patient data recording and viewing system in the course of surgery

From microscope to head-mounted display: integrating hand tracking into microsurgical augmented reality

Evaluation of User‐Interfaces for Controlling Movements of Virtual Minimally Invasive Surgical Instruments

Gaze gesture based human robot interaction for laparoscopic surgery

Gaze Contingent Cartesian Control of a Robotic Arm for Laparoscopic Surgery.

A New Precise Contactless Medical Image Multimodal Interaction System for Surgical Practice.

Gaze Contingent Control for an Articulated Mechatronic Laparoscope

Sensory Glove-Based Surgical Robot User Interface

Hand gesture based control with multi-modality data - towards surgical applications

Unconstrained lightweight control interface for robot-assisted minimally invasive surgery using MediaPipe framework and head-mounted display

Gaze-Contingent Motor Channelling, Haptic Constraints and Associated Cognitive Demand for Robotic MIS

Biomechanics-machine learning system for surgical gesture analysis and development of technologies for minimal access surgery

Gaze-Contingent Motor Channelling and Haptic Constraints for Minimally Invasive Robotic Surgery

Tracking Hand Hygiene Gestures with Leap Motion Controller

Computerized Medical Imaging and Graphics

Workflow efficiency pilot study of Surgery Viewer©: A secure hands-free intraoperative multimedia interface for Google Glass™

Touchless Control of Picture Archiving and Communication System in Operating Room Environment: A Comparative Study of Input Methods

Coronary Heart Disease Preoperative Gesture Interactive Diagnostic System Based on Augmented Reality.