Abstract:Significance: Minimally invasive surgery (MIS) has shown vast improvement over open surgery by reducing post-operative stays, intraoperative blood loss, and infection rates. However, in spite of these improvements, there are still prevalent issues surrounding MIS that may be addressed through hyperspectral imaging (HSI). We present a laparoscopic HSI system to further advance the field of MIS. Aim: We present an imaging system that integrates high-speed HSI technology with a clinical laparoscopic setup and validate the system's accuracy and functionality. Different configurations that cover the visible (VIS) to near-infrared (NIR) range of electromagnetism are assessed by gauging the spectral fidelity and spatial resolution of each hyperspectral camera. Approach: Standard Spectralon reflectance tiles were used to provide ground truth spectral footprints to compare with those acquired by our system using the root mean squared error (RMSE). Demosaicing techniques were investigated and used to measure and improve spatial resolution, which was assessed with a USAF resolution test target. A perception-based image quality evaluator was used to assess the demosaicing techniques we developed. Two configurations of the system were developed for evaluation. The functionality of the system was investigated in a phantom study and by imaging ex vivo tissues. Results: Multiple configurations of our system were tested, each covering different spectral ranges, including VIS (460 to 600 nm), red/NIR (RNIR) (610 to 850 nm), and NIR (665 to 950 nm). Each configuration is capable of achieving real-time imaging speeds of up to 20 frames per second. RMSE values of 3.51 ± 2.03 % , 3.43 ± 0.84 % , and 3.47% were achieved for the VIS, RNIR, and NIR systems, respectively. We obtained sub-millimeter resolution using our demosaicing techniques. Conclusions: We developed and validated a high-speed hyperspectral laparoscopic imaging system. The HSI system can be used as an intraoperative imaging tool for tissue classification during laparoscopic surgery.

Deep learning approach for hyperspectral image demosaicking, spectral correction and high-resolution RGB reconstruction

A self-supervised and adversarial approach to hyperspectral demosaicking and RGB reconstruction in surgical imaging

Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imaging

A Hyperspectral Surgical Microscope with Super-Resolution Reconstruction for Intraoperative Image Guidance

RGB to Hyperspectral: Spectral Reconstruction for Enhanced Surgical Imaging

Hyperspectral imaging with deep learning for quantification of tissue hemoglobin, melanin, and scattering

Spatial and Spectral Reconstruction of Breast Lumpectomy Hyperspectral Images

Hyperspectral Imaging from a Raw Mosaic Image with End-to-end Learning

Dual-camera Laparoscopic Imaging with Super-Resolution Reconstruction for Intraoperative Hyperspectral Image Guidance

Hyperspectral Image Reconstruction Based on Improved Residual Dense Network

Deep Learning-Based Correction and Unmixing of Hyperspectral Images for Brain Tumor Surgery

Convolutional neural network advances in demosaicing for fluorescent cancer imaging with color-near-infrared sensors

mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

Deep intra-operative illumination calibration of hyperspectral cameras

A Snapshot Near-Infrared Hyperspectral Demosaicing Method with Convolutional Neural Networks in Low Illumination Environment

High-Throughput Deep Learning Microscopy Using Multi-Angle Super-Resolution

Intra-operative Brain Tumor Detection with Deep Learning-Optimized Hyperspectral Imaging

Hyperspectral demosaicking and crosstalk correction using deep learning

Efficient and Accurate Hyperspectral Image Demosaicing with Neural Network Architectures

Design and validation of a high-speed hyperspectral laparoscopic imaging system

Unsupervised Super Resolution Network for Hyperspectral Histologic Imaging