Abstract:Large DOF (depth-of-field) with high SNR (signal-noise-ratio) imaging is a crucial technique for applications from security monitoring to medical diagnostics. However, traditional optical design for large DOF requires a reduction in aperture size, and hence with a decrease in light throughput and SNR. In this paper, we report a computational imaging system integrating dual-aperture optics with a physics-informed dual-encoder neural network to realize prominent DOF extension. Boosted by human vision mechanism and optical imaging law, the dual-aperture imaging system is consisted of a small-aperture NIR camera to provide sharp edge and a large-aperture VIS camera to provide faithful color. To solve the imaging inverse problem in NIR-VIS fusion with different apertures, a specific network with parallel double encoders and the multi-scale fusion module is proposed to adaptively extract and learn the useful features, which contributes to preventing color deviation while preserving delicate scene textures. The proposed imaging framework is flexible and can be designed in different protos with varied optical elements for different applications. We provide theory for system design, demonstrate a prototype device, establish a real-scene dataset containing 3000 images, perform elaborate ablation studies and conduct peer comparative experiments. The experimental results demonstrate that our method effectively produces high-fidelity with larger DOF range than input raw images about 3 times. Without complex optical design and strict practical limitations, this novel, intelligent and integratable system is promising for variable vision applications such as smartphone photography, computational measurement, and medical imaging.

Passive Snapshot Coded Aperture Dual-Pixel RGB-D Imaging

DoCam: Depth Sensing with an Optical Image Stabilization Supported RGB Camera.

Simultaneous Coded Aperture and Dictionary Optimization in Compressive Spectral Imaging Via Coherence Minimization.

Compressive Single-Pixel Hyperspectral Imaging Using RGB Sensors

Hybrid Compressive Spectral Imaging with Coded Aperture Optimization Based on Coherence Minimization

39 000-Subexposures/s Dual-ADC CMOS Image Sensor With Dual-Tap Coded-Exposure Pixels for Single-Shot HDR and 3-D Computational Imaging

Snapshot Compressive ToF+Spectral Imaging via Optimized Color-Coded Apertures

Dual-camera Design for Coded Aperture Snapshot Spectral Imaging.

A 39,000 Subexposures/s CMOS Image Sensor with Dual-tap Coded-exposure Data-memory Pixel for Adaptive Single-shot Computational Imaging

Deep Learning Enabled Reflective Coded Aperture Snapshot Spectral Imaging

DMDC: Dynamic-mask-based dual camera design for snapshot Hyperspectral Imaging

Single-Photon 3D Imaging with Equi-Depth Photon Histograms

Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restoration

Double-channel compressive spectral imaging via complementary code patterns

End-to-end Learned, Optically Coded Super-resolution SPAD Camera

DMDC: a cross-attention network for dynamic mask-based dual-camera snapshot hyperspectral Photography

A 1920 $\times$ 1080 Array 2-D/3-D Image Sensor with 3- $\mu$ S Row-Time Single-Slope ADC and 100-Mhz Demodulated PPD Locked-In Pixel

Subpixel Three Dimensional Imaging Using Two Dimensional Optical Encoding with a Downscaled Avalanche Photodiode Array

Design of on-chip coded high-resolution 2D imaging via 3D compressed sensing

Passive Inter-Photon Imaging

Large Depth-of-field Computational Imaging with Multi-Spectral and Dual-Aperture Optics.