Abstract:Computational optical imaging (COI) systems leverage optical coding elements (CE) in their setups to encode a high-dimensional scene in a single or multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Conventional approaches rely on random patterns or analytical designs to set the distribution of the CE. However, the available data and algorithm capabilities of deep neural networks (DNNs) have opened a new horizon in CE data-driven designs that jointly consider the optical encoder and computational decoder. Specifically, by modeling the COI measurements through a fully differentiable image formation model that considers the physics-based propagation of light and its interaction with the CEs, the parameters that define the CE and the computational decoder can be optimized in an end-to-end (E2E) manner. Moreover, by optimizing just CEs in the same framework, inference tasks can be performed from pure optics. This work surveys the recent advances on CE data-driven design and provides guidelines on how to parametrize different optical elements to include them in the E2E framework. Since the E2E framework can handle different inference applications by changing the loss function and the DNN, we present low-level tasks such as spectral imaging reconstruction or high-level tasks such as pose estimation with privacy preserving enhanced by using optimal task-based optical architectures. Finally, we illustrate classification and 3D object recognition applications performed at the speed of the light using all-optics DNN.

Deep Learning for Sensing Matrix Prediction in Computational Microwave Imaging With Coded-Apertures

Optimal Phase Mask Design for Coherent Modulation Imaging by Deep Learning

Simultaneous Coded Aperture and Dictionary Optimization in Compressive Spectral Imaging Via Coherence Minimization.

Convolutional Neural Network Based Computational Imaging Spectroscopy

Large-aperture computational single-sensor microwave imager using 1-bit programmable coding metasurface at single frequency

Deep Coded Aperture Design: An End-to-End Approach for Computational Imaging Tasks

Deep Blind Compressed Sensing

Towards Real-Time Through-Obstacle Imaging Based on Compressed Sensing for Sparse Objects

Joint Learning-Based Coded Aperture Compressive Temporal Imaging

Coded aperture compressive temporal imaging via unsupervised lightweight local-global networks with geometric characteristics

Design of on-chip coded high-resolution 2D imaging via 3D compressed sensing

Coded Aperture Compression Temporal Imaging Based on a Dual-Mask and Deep Denoiser

Deep Network for Image Compressed Sensing Coding Using Local Structural Sampling

Separable compressed coded aperture imaging via singular value decomposition

Deep Coding Patterns Design for Compressive Near-Infrared Spectral Classification

Source Coding Based Millimeter-Wave Channel Estimation with Deep Learning Based Decoding

Scene-adaptive Coded Apertures Imaging

Microwave Imaging by Deep Learning Network: Feasibility and Training Method

Deep Optical Coding Design in Computational Imaging

Deep Sparse Coding Using Optimized Linear Expansion of Thresholds