Abstract:Deep learning (DL) is a powerful tool in computational imaging for many applications. A common strategy is to reconstruct a preliminary image as the input of a neural network to achieve an optimized image. Usually, the preliminary image is acquired with the prior knowledge of the imaging model. One outstanding challenge, however, is the degree to which the actual imaging model deviates from the assumed model. Model mismatches degrade the quality of the preliminary image and therefore affect the DL predictions. Another main challenge is that since most imaging inverse problems are ill-posed and the networks are over-parameterized, DL networks have flexibility to extract features from the data that are not directly related to the imaging model. To solve these challenges, a two-step-training DL (TST-DL) framework is proposed for real-time computational imaging without physics priors. First, a single fully-connected layer (FCL) is trained to directly learn the model. Then, this FCL is fixed and concatenated with an un-trained U-Net architecture for a second-step training to improve the output image fidelity, resulting in four main advantages. First, it does not rely on an accurate representation of the imaging model since the model is directly learned. Second, real-time imaging can be achieved. Third, the TST-DL network is trained in the desired direction and the predictions are improved since the first step is constrained to learn the model and the second step improves the result by learning the optimal regularizer. Fourth, the approach accommodates any size and dimensionality of data. We demonstrate this framework using a linear single-pixel camera imaging model. The results are quantitatively compared with those from other DL frameworks and model-based iterative optimization approaches. We further extend this concept to nonlinear models in the application of image de-autocorrelation.

Deep network series for large-scale high-dynamic range imaging

Cascaded Deep Networks with Multiple Receptive Fields for Infrared Image Super-Resolution

The R2D2 deep neural network series paradigm for fast precision imaging in radio astronomy

Scalable Non-Cartesian Magnetic Resonance Imaging with R2D2

Multi-Scale Dense Networks for Deep High Dynamic Range Imaging

Application of a Neural Network to Store and Compute the Optical Properties of Non-Spherical Particles

Deep SR-HDR: Joint Learning of Super-Resolution and High Dynamic Range Imaging for Dynamic Scenes

Deep learning in computational imaging

High Dynamic Range Imaging for Dynamic Scenes With Large-Scale Motions and Severe Saturation

ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content

A Two-step-training Deep Learning Framework for Real-time Computational Imaging without Physics Priors

Deep high dynamic range imaging of dynamic scenes

Deep HDR Imaging via A Non-local Network

DeepNIS: Deep Neural Network for Nonlinear Electromagnetic Inverse Scattering

Deep Residual Neural Network Design for Super-Resolution Imaging.

Large Field Astronomical Image Restoration and Super-resolution Reconstruction Using Deep Learning

High dynamic range infrared image compression and denoising

Deep Progressive Feature Aggregation Network for High Dynamic Range Imaging

UPDResNN: A Deep Light-Weight Image Upsampling and Deblurring Residual Neural Network

Deep Non-line-of-sight Imaging from Under-scanning Measurements.