Abstract:Controllable Depth-of-Field (DoF) imaging commonly produces amazing visual effects based on heavy and expensive high-end lenses. However, confronted with the increasing demand for mobile scenarios, it is desirable to achieve a lightweight solution with Minimalist Optical Systems (MOS). This work centers around two major limitations of MOS, i.e., the severe optical aberrations and uncontrollable DoF, for achieving single-lens controllable DoF imaging via computational methods. A Depth-aware Controllable DoF Imaging (DCDI) framework is proposed equipped with All-in-Focus (AiF) aberration correction and monocular depth estimation, where the recovered image and corresponding depth map are utilized to produce imaging results under diverse DoFs of any high-end lens via patch-wise convolution. To address the depth-varying optical degradation, we introduce a Depth-aware Degradation-adaptive Training (DA2T) scheme. At the dataset level, a Depth-aware Aberration MOS (DAMOS) dataset is established based on the simulation of Point Spread Functions (PSFs) under different object distances. Additionally, we design two plug-and-play depth-aware mechanisms to embed depth information into the aberration image recovery for better tackling depth-aware degradation. Furthermore, we propose a storage-efficient Omni-Lens-Field model to represent the 4D PSF library of various lenses. With the predicted depth map, recovered image, and depth-aware PSF map inferred by Omni-Lens-Field, single-lens controllable DoF imaging is achieved. Comprehensive experimental results demonstrate that the proposed framework enhances the recovery performance, and attains impressive single-lens controllable DoF imaging results, providing a seminal baseline for this field. The source code and the established dataset will be publicly available at <a class="link-external link-https" href="https://github.com/XiaolongQian/DCDI" rel="external noopener nofollow">this https URL</a>.

Investigating deep optics model representation in affecting resolved all-in-focus image quality and depth estimation fidelity

Calibration-free Deep Optics for Depth Estimation with Precise Simulation

Optimal Phase Mask Design for Coherent Modulation Imaging by Deep Learning

Modeling the Depth-Sectioning Effect in Reflection-Mode Dynamic Speckle-Field Interferometric Microscopy.

4-K-resolution Minimalist Optical System Design Based on Deep Learning.

Deep Tissue Focusing Based on Machine Learning

Reliability of Wavefront Shaping Based on Coherent Optical Adaptive Technique in Deep Tissue Focusing.

High-performance Imaging with an Advanced Non-Imaging Lens Based on Full-Path Optical Diffraction Calculation in Two-Dimensional Space

Imaging Simulation and Restoration for Mobile-based Long-focus Camera with Optical Stabilization

Deep Optics for Monocular Depth Estimation and 3D Object Detection

A simulation imaging method for actual optical system and results analysis

Simplified design method for optical imaging systems based on deep learning

Towards Single-Lens Controllable Depth-of-Field Imaging via All-in-Focus Aberration Correction and Monocular Depth Estimation

Computational Optics for Mobile Terminals in Mass Production

Quality Optimization and Control Technology for A Full Field-Of-View Imaging Surface Using A Stationary Phase Method and an Ambiguity Function Method

Deep learning in optics and photonics for statistical inference, computing, and inverse design

Deformable Mirror Based Optimal PSF Engineering for 3D Super-Resolution Imaging

Large depth-of-field ultra-compact microscope by progressive optimization and deep learning

End-to-End Hybrid Refractive-Diffractive Lens Design with Differentiable Ray-Wave Model

Inverse designed extended depth of focus meta-optics for broadband imaging in the visible

Integrative Optimization of the Practical Wavefront Coding Systems for Depth-Of-Field Extension