Abstract:Significance: Deep learning enables label-free all-optical biopsies and automated tissue classification. Endoscopic systems provide intraoperative diagnostics to deep tissue and speed up treatment without harmful tissue removal. However, conventional multi-core fiber (MCF) endoscopes suffer from low resolution and artifacts, which hinder tumor diagnostics.Aim: We introduce a method to enable unpixelated, high-resolution tumor imaging through a given MCF with a diameter of around 0.65 mm and arbitrary core arrangement and inhomogeneous transmissivity.Approach: Image reconstruction is based on deep learning and the digital twin concept of the single-reference-based simulation with inhomogeneous optical properties of MCF and transfer learning on a small experimental dataset of biological tissue. The reference provided physical information about the MCF during the training processes.Results: For the simulated data, hallucination caused by the MCF inhomogeneity was eliminated, and the averaged peak signal-to-noise ratio and structural similarity were increased from 11.2 dB and 0.20 to 23.4 dB and 0.74, respectively. By transfer learning, the metrics of independent test images experimentally acquired on glioblastoma tissue <i>ex vivo</i> can reach up to 31.6 dB and 0.97 with 14 fps computing speed.Conclusions: With the proposed approach, a single reference image was required in the pre-training stage and laborious acquisition of training data was bypassed. Validation on glioblastoma cryosections with transfer learning on only 50 image pairs showed the capability for high-resolution deep tissue retrieval and high clinical feasibility.

Fiber Bundle Imaging Resolution Enhancement Using Deep Learning.

Deep Learning Enhanced Fluorescence Emission Difference Microscopy

Enhanced Stimulated Raman and Fluorescence Imaging by Single-Frame Trained BDN

Spatially Variant Deblur and Image Enhancement in a Single Multimode Fiber Imaged by Deep Learning

Model-based deep learning for fiber bundle infrared image restoration

Learning-based High-Resolution Lensless Fiber Bundle Imaging for Tumor

Universal and High-Fidelity Resolution Extending for Fluorescence Microscopy Using a Single-Training Physics-Informed Sparse Neural Network

Feature-enhanced Fiber Bundle Imaging Based on Light Field Acquisition

Enhancing fluorescence microscopy performance with Richardson-Lucy based deep learning

Resolution Enhancement for Flexible Microscopic Imaging Based on Dictionary Learning.

All-fiber high-speed image detection enabled by deep learning

Incorporating the Image Formation Process into Deep Learning Improves Network Performance

Optical Fiber Bundle-Based High-Speed and Precise Micro-Scanning for Image High-Resolution Reconstruction

High-Speed All-Fiber Micro-Imaging with Large Depth of Field

Resolution-enhanced multi-core fiber imaging learned on a digital twin for cancer diagnosis

Fiber Bundle Shifting Endomicroscopy for High-Resolution Imaging.

Learned End-to-end High-Resolution Lensless Fiber Imaging Towards Real-Time Cancer Diagnosis

Learned end-to-end high-resolution lensless fiber imaging toward intraoperative real-time cancer diagnosis

All-fiber Ultrafast Image Detection Enabled by Deep Learning

Deep Learning Microscopy

Deep Learning Captures More Accurate Diffusion Fiber Orientations Distributions than Constrained Spherical Deconvolution