Abstract:Relying on either deep models or physical models are two mainstream approaches for solving inverse sample reconstruction problems in programmable illumination computational microscopy. Solutions based on physical models possess strong generalization capabilities while struggling with global optimization of inverse problems due to a lack of insufficient physical constraints. In contrast, deep learning methods have strong problem-solving abilities, but their generalization ability is often questioned because of the unclear physical principles. Besides, conventional deep models are difficult to apply to some specific scenes because of the difficulty in acquiring high-quality training data and their limited capacity to generalize across different scenarios. In this paper, to combine the advantages of deep models and physical models together, we propose a hybrid framework consisting of three sub-neural networks (two deep learning networks and one physics-based network). We first obtain a result with rich semantic information through a light deep learning neural network and then use it as the initial value of the physical network to make its output comply with physical process constraints. These two results are then used as the input of a fusion deep learning neural work which utilizes the paired features between the reconstruction results of two different models to further enhance imaging quality. The final result integrates the advantages of both deep models and physical models and can quickly solve the computational reconstruction inverse problem in programmable illumination computational microscopy and achieve better results. We verified the feasibility and effectiveness of the proposed hybrid framework with theoretical analysis and actual experiments on resolution targets and biological samples.

Simulation-driven Learning: a Deep Learning Approach for Image Scanning Microscopy Via Physical Imaging Models

Deep Learning Enhanced Fluorescence Emission Difference Microscopy

Deep Learning Virtual Colorization Overcoming Chromatic Aberrations in Singlet Lens Microscopy

Wavefront Reconstruction Based on Deep Transfer Learning for Microscopy.

Adaptive Optics For Structured Illumination Microscopy Based On Deep Learning

Universal and High-Fidelity Resolution Extending for Fluorescence Microscopy Using a Single-Training Physics-Informed Sparse Neural Network

Deep Learning Virtual Zernike Phase Contrast Imaging for Singlet Microscopy

Deep-Learning-Based Image Reconstruction and Enhancement in Optical Microscopy

Two-stage Generative Models of Simulating Training Data at the Voxel Level for Large-Scale Microscopy Bioimage Segmentation

Learning from Simulation: an End-to-end Deep-Learning Approach for Computational Ghost Imaging

Deep learning-enhanced fluorescence microscopy via confocal physical imaging model.

A Hybrid Reconstruction of the Physical Model with the Deep-Learning That Improves Structured Illumination Microscopy

Hybrid deep learning and physics-based neural network for programmable illumination computational microscopy

Super-resolution reconstruction of structured illumination microscopy using deep-learning and sparse deconvolution

A Survey on Applications of Deep Learning in Microscopy Image Analysis

Robust single-shot 3D fluorescence imaging in scattering media with a simulator-trained neural network

Improving Reconstruction of Structured Illumination Microscopy Images Via Dual-Domain Learning

Evaluation and development of deep neural networks for image super-resolution in optical microscopy

A cascaded deep network for reconstruction of structured illumination microscopy

Deep‐MSIM: Fast Image Reconstruction with Deep Learning in Multifocal Structured Illumination Microscopy

Deep Learning Microscopy