Abstract:Large DOF (depth-of-field) with high SNR (signal-noise-ratio) imaging is a crucial technique for applications from security monitoring to medical diagnostics. However, traditional optical design for large DOF requires a reduction in aperture size, and hence with a decrease in light throughput and SNR. In this paper, we report a computational imaging system integrating dual-aperture optics with a physics-informed dual-encoder neural network to realize prominent DOF extension. Boosted by human vision mechanism and optical imaging law, the dual-aperture imaging system is consisted of a small-aperture NIR camera to provide sharp edge and a large-aperture VIS camera to provide faithful color. To solve the imaging inverse problem in NIR-VIS fusion with different apertures, a specific network with parallel double encoders and the multi-scale fusion module is proposed to adaptively extract and learn the useful features, which contributes to preventing color deviation while preserving delicate scene textures. The proposed imaging framework is flexible and can be designed in different protos with varied optical elements for different applications. We provide theory for system design, demonstrate a prototype device, establish a real-scene dataset containing 3000 images, perform elaborate ablation studies and conduct peer comparative experiments. The experimental results demonstrate that our method effectively produces high-fidelity with larger DOF range than input raw images about 3 times. Without complex optical design and strict practical limitations, this novel, intelligent and integratable system is promising for variable vision applications such as smartphone photography, computational measurement, and medical imaging.

Extending the depth of field of imaging systems using depth sensing camera

Multi-image-distance Imaging System for Extending Depth-of-field

Joint Focusing Method Based on Optimal Image-Distance Interval for Multi-Image-distance Imaging System

Extended Depth of Field Reconstruction with Complex Field Estimation

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

Extended Depth of Field Integral Imaging Using Multi-Focus Fusion

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

Depth Extension For Multiple Focused Images By Adaptive Point Spread Function

DLP-Fusion: Depth of Field, Light Source, and Polarization Fusion Towards Intelligent Optical Imaging for Complex Scenes

A Fused Estimation Method of Depth Based on Depth from Stereo and Depth from Focus

High quality depth maps from stereo matching and ToF camera.

Depth of Field Extension for Microscopic Structured Light Vision System Based on Image Fusion

Extending Depth of Field for Varifocal Multiview Images

Depth map enhancement based on color and depth consistency

Large Depth-of-field Computational Imaging with Multi-Spectral and Dual-Aperture Optics.

Depth-of-field Extension in Integral Imaging Using Multi-Focus Elemental Images.

Construction of all-in-focus images assisted by depth sensing

Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios

High Quality Multi-Focus Image Fusion Using Self-Similarity and Depth Information

Depth-of-field expansion method based on multidimensional structure and edge-guided correction

Depth Fused from Intensity Range and Blur Estimation for Light-Field Cameras