Abstract:Currently, surgeons in operating rooms are forced to focus their attention both on the patient’s body and on flat low-quality surgical monitors, in order to get all the information needed to successfully complete surgeries. The way the data are displayed leads to disturbances of the surgeon’s visuals, which may affect his performances, besides the fact that other members of the surgical team do not have proper visual tools able to aid him. The idea underlying this paper is to exploit mixed reality to support surgeons during surgical procedures. In particular, the proposed experimental setup, employed in the operating room, is based on an architecture that put together the Microsoft HoloLens, a Digital Imaging and Communications in Medicine (DICOM) player and a mixed reality visualization tool (i.e., Spectator View) developed by using the Mixed Reality Toolkit in Unity with Windows 10 SDK. The suggested approach enables visual information on the patient’s body as well as information on the results of medical screenings to be visualized on the surgeon’s headsets. Additionally, the architecture enables any data and details to be shared by the team members or by external users during surgical operations. The paper analyses in detail advantages and drawbacks that the surgeons have found when they wore the Microsoft HoloLens headset during all the ten open abdomen surgeries conducted at the IRCCS Hospital “Giovanni Paolo II” in the city of Bari (Italy). A survey based on Likert scale demonstrates how the use of the suggested tools can increase the execution speed by allowing multitasking procedures, i.e., by checking medical images at high resolution without leaving the operating table and the patient. On the other hand, the survey also reveals an increase in the physical stress and reduced comfort due to the weight of the Microsoft HoloLens device, along with drawbacks due to the battery autonomy. Additionally, the survey seems to encourage the use of DICOM Viewer and Spectator View both for surgical education and for improving surgery outcomes. Note that the real use of the conceived platform in the operating room represents a remarkable feature of this paper, since most if not all the studies conducted so far in literature exploit mixed reality only in simulated environments and not in real operating rooms. In conclusion, the study clearly highlights that, despite the challenges required in the forthcoming years to improve the current technology, mixed reality represents a promising technique that will soon enter the operating rooms to support surgeons during surgical procedures in many hospitals across the world.

A novel training session: mixed reality evisceration surgery webinar

A mixed reality framework for microsurgery simulation with visual-tactile perception

Can mixed reality technologies teach surgical skills better than traditional methods? A prospective randomised feasibility study

Advanced liver surgery training in collaborative VR environments

Towards Augmented Reality-based Suturing in Monocular Laparoscopic Training

Virtual reality training in neurosurgery: Review of current status and future applications

New Technologies for the Surgical Curriculum

979 Enhancing Surgical Training Through Augmented Reality: A Pilot Study on the Use of the Microsoft HoloLens for Live-Streaming from Theatre

Stand in surgeon's shoes: virtual reality cross-training to enhance teamwork in surgery

Augmented Reality Microsurgery: Proof of Concept for a Novel Approach to Microsurgical Field Visualization in Plastic Surgery

VIS-a-VE: Visual Augmentation for Virtual Environments in Surgical Training

Surgical training with phantoms in strabismus surgery

Virtual Reality Training and Assessment in Laparoscopic Rectum Surgery

A novel augmented reality simulator for skills assessment in minimal invasive surgery

Experimental Setup Employed in the Operating Room Based on Virtual and Mixed Reality: Analysis of Pros and Cons in Open Abdomen Surgery

Virtual reality for synergistic surgical training and data generation

Mixed Reality in Visceral Surgery: Development of a Suitable Workflow and Evaluation of Intraoperative Use-cases

Mixed reality applications in upper extremity surgery: the future is now

Immersive teaching using virtual reality technology to improve ophthalmic surgical skills for medical postgraduate students

Surgical Training Using Augmented Reality-Based Optical Head-Mounted Displays: A Pilot Study

From Augmented to Virtual Reality in Plastic Surgery: Blazing the Trail to a New Frontier