Abstract:Relying on either deep models or physical models are two mainstream approaches for solving inverse sample reconstruction problems in programmable illumination computational microscopy. Solutions based on physical models possess strong generalization capabilities while struggling with global optimization of inverse problems due to a lack of insufficient physical constraints. In contrast, deep learning methods have strong problem-solving abilities, but their generalization ability is often questioned because of the unclear physical principles. Besides, conventional deep models are difficult to apply to some specific scenes because of the difficulty in acquiring high-quality training data and their limited capacity to generalize across different scenarios. In this paper, to combine the advantages of deep models and physical models together, we propose a hybrid framework consisting of three sub-neural networks (two deep learning networks and one physics-based network). We first obtain a result with rich semantic information through a light deep learning neural network and then use it as the initial value of the physical network to make its output comply with physical process constraints. These two results are then used as the input of a fusion deep learning neural work which utilizes the paired features between the reconstruction results of two different models to further enhance imaging quality. The final result integrates the advantages of both deep models and physical models and can quickly solve the computational reconstruction inverse problem in programmable illumination computational microscopy and achieve better results. We verified the feasibility and effectiveness of the proposed hybrid framework with theoretical analysis and actual experiments on resolution targets and biological samples.

Lumiproxy: A Hybrid Representation of Image-Based Models

Modeling Based On Multi-Scale Quadtree Image

View-Dependent Rendering of Virtual Plane Models

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Image-Space Texture-Based Output-Coherent Surface Flow Visualization.

Lighting and Shadow Interpolation Using Intrinsic Lumigraphs

Output-coherent image-space LIC for surface flow visualization

Real-time Rendering Algorithm Based on a Hybrid Rendering Scheme

Walking into Images: Virtual Plane Mosaics for Plenoptic Modeling

Bernoulli Embedding Model and Its Application in Texture Mapping

Rendering of Virtual Environments Based on Polygonal & Point-Based Models

Spatial pooling of heterogeneous features for image applications.

Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstruction

P-quadtrees: a Point and Polygon Hybrid Multi-Resolution Rendering Approach

Hybrid deep learning and physics-based neural network for programmable illumination computational microscopy

A Lightweight Hybrid Representation for Virtual Complex Scenes

Volumetric Stereo and Silhouette Fusion for Image-Based Modeling

Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction

Hybrid Forward Resampling and Volume Rendering

Sparse representation of images with hybrid linear models

Intrinsic Image Decomposition Using Point Cloud Representation