Abstract:Deep learning (DL) is a powerful tool in computational imaging for many applications. A common strategy is to reconstruct a preliminary image as the input of a neural network to achieve an optimized image. Usually, the preliminary image is acquired with the prior knowledge of the imaging model. One outstanding challenge, however, is the degree to which the actual imaging model deviates from the assumed model. Model mismatches degrade the quality of the preliminary image and therefore affect the DL predictions. Another main challenge is that since most imaging inverse problems are ill-posed and the networks are over-parameterized, DL networks have flexibility to extract features from the data that are not directly related to the imaging model. To solve these challenges, a two-step-training DL (TST-DL) framework is proposed for real-time computational imaging without physics priors. First, a single fully-connected layer (FCL) is trained to directly learn the model. Then, this FCL is fixed and concatenated with an un-trained U-Net architecture for a second-step training to improve the output image fidelity, resulting in four main advantages. First, it does not rely on an accurate representation of the imaging model since the model is directly learned. Second, real-time imaging can be achieved. Third, the TST-DL network is trained in the desired direction and the predictions are improved since the first step is constrained to learn the model and the second step improves the result by learning the optimal regularizer. Fourth, the approach accommodates any size and dimensionality of data. We demonstrate this framework using a linear single-pixel camera imaging model. The results are quantitatively compared with those from other DL frameworks and model-based iterative optimization approaches. We further extend this concept to nonlinear models in the application of image de-autocorrelation.

A unified framework of deep unfolding for compressed color imaging

Superpixel-Based Linear Reconstruction Method for Dual-Camera Compressed Hyperspectral Imaging System

Compressive Single-Pixel Hyperspectral Imaging Using RGB Sensors

Image reconstruction based on back propagation learning in Compressed Sensing theory

A Unified Active and Semi-Supervised Learning Framework for Image Compression

Color Image Compression with Transform Domain Down-Sampling and Deep Convolutional Reconstruction

Deep-learning-based Single-Photon-counting Compressive Imaging Via Jointly Trained Subpixel Convolution Sampling.

Single Photon Counting Compressive Imaging Based on a Sampling and Reconstruction Integrated Deep Network

Latent Diffusion Prior Enhanced Deep Unfolding for Snapshot Spectral Compressive Imaging

High-fidelity image reconstruction for compressed ultrafast photography via an augmented-Lagrangian and deep-learning hybrid algorithm

Unified and Scalable Deep Image Compression Framework for Human and Machine

Image Reconstruction for Compressed Ultrafast Photography Based on Manifold Learning and Alternating Direction Method of Multipliers

A Two-step-training Deep Learning Framework for Real-time Computational Imaging without Physics Priors

Low-Light Color Imaging via Dual Camera Acquisition.

Color computational ghost imaging by deep learning based on simulation data training

Anti-noise computational imaging using unsupervised deep learning

CoCoCs: Co-Optimized Compressive Imaging Driven by High-Level Vision

Dynamic Proximal Unrolling Network for Compressive Imaging

Real-time physical compression computational ghost imaging based on array spatial light field modulation and deep learning

Deep Physics-Guided Unrolling Generalization for Compressed Sensing

Multi-image super-resolution based low complexity deep network for image compressive sensing reconstruction