Abstract:Deep learning (DL) is a powerful tool in computational imaging for many applications. A common strategy is to reconstruct a preliminary image as the input of a neural network to achieve an optimized image. Usually, the preliminary image is acquired with the prior knowledge of the imaging model. One outstanding challenge, however, is the degree to which the actual imaging model deviates from the assumed model. Model mismatches degrade the quality of the preliminary image and therefore affect the DL predictions. Another main challenge is that since most imaging inverse problems are ill-posed and the networks are over-parameterized, DL networks have flexibility to extract features from the data that are not directly related to the imaging model. To solve these challenges, a two-step-training DL (TST-DL) framework is proposed for real-time computational imaging without physics priors. First, a single fully-connected layer (FCL) is trained to directly learn the model. Then, this FCL is fixed and concatenated with an un-trained U-Net architecture for a second-step training to improve the output image fidelity, resulting in four main advantages. First, it does not rely on an accurate representation of the imaging model since the model is directly learned. Second, real-time imaging can be achieved. Third, the TST-DL network is trained in the desired direction and the predictions are improved since the first step is constrained to learn the model and the second step improves the result by learning the optimal regularizer. Fourth, the approach accommodates any size and dimensionality of data. We demonstrate this framework using a linear single-pixel camera imaging model. The results are quantitatively compared with those from other DL frameworks and model-based iterative optimization approaches. We further extend this concept to nonlinear models in the application of image de-autocorrelation.

Learning from Simulation: an End-to-end Deep-Learning Approach for Computational Ghost Imaging

Computational Ghost Imaging in Scattering Media Using Simulation-Based Deep Learning

Computational Ghost Imaging Via Adaptive Deep Dictionary Learning.

Computational ghost imaging with deep compressed sensing

Deep-learning-based Ghost Imaging.

Enhanced Deep Learning Approach Based on the Deep Convolutional Encoder–Decoder Architecture for Electromagnetic Inverse Scattering Problems

A residual-based deep learning approach for ghost imaging.

Computational ghost imaging using deep learning

Extendible Ghost Imaging with High Reconstruction Quality in Strong Scattering Medium

Simulation-driven Learning: a Deep Learning Approach for Image Scanning Microscopy Via Physical Imaging Models

Sub-Nyquist Computational Ghost Imaging with Deep Learning

DeepGhost: Real-Time Computational Ghost Imaging Via Deep Learning

Color computational ghost imaging by deep learning based on simulation data training

A Two-step-training Deep Learning Framework for Real-time Computational Imaging without Physics Priors

Deep-learning denoising computational ghost imaging

Deep learning in computational imaging

Color Ghost Imaging Through a Dynamic Scattering Medium Based on Deep Learning

High-speed computational ghost imaging with compressed sensing based on a convolutional neural network

End-to-end Computational Ghost Imaging Method That Suppresses Atmospheric Turbulence.

Far-field super-resolution ghost imaging with a deep neural network constraint

Ghost Imaging by Single-Layer Neural Network Based on Forward Physical Model