Abstract:Controllable Depth-of-Field (DoF) imaging commonly produces amazing visual effects based on heavy and expensive high-end lenses. However, confronted with the increasing demand for mobile scenarios, it is desirable to achieve a lightweight solution with Minimalist Optical Systems (MOS). This work centers around two major limitations of MOS, i.e., the severe optical aberrations and uncontrollable DoF, for achieving single-lens controllable DoF imaging via computational methods. A Depth-aware Controllable DoF Imaging (DCDI) framework is proposed equipped with All-in-Focus (AiF) aberration correction and monocular depth estimation, where the recovered image and corresponding depth map are utilized to produce imaging results under diverse DoFs of any high-end lens via patch-wise convolution. To address the depth-varying optical degradation, we introduce a Depth-aware Degradation-adaptive Training (DA2T) scheme. At the dataset level, a Depth-aware Aberration MOS (DAMOS) dataset is established based on the simulation of Point Spread Functions (PSFs) under different object distances. Additionally, we design two plug-and-play depth-aware mechanisms to embed depth information into the aberration image recovery for better tackling depth-aware degradation. Furthermore, we propose a storage-efficient Omni-Lens-Field model to represent the 4D PSF library of various lenses. With the predicted depth map, recovered image, and depth-aware PSF map inferred by Omni-Lens-Field, single-lens controllable DoF imaging is achieved. Comprehensive experimental results demonstrate that the proposed framework enhances the recovery performance, and attains impressive single-lens controllable DoF imaging results, providing a seminal baseline for this field. The source code and the established dataset will be publicly available at <a class="link-external link-https" href="https://github.com/XiaolongQian/DCDI" rel="external noopener nofollow">this https URL</a>.

Deep Optics for Monocular Depth Estimation and 3D Object Detection

Calibration-free Deep Optics for Depth Estimation with Precise Simulation

Investigating deep optics model representation in affecting resolved all-in-focus image quality and depth estimation fidelity

4-K-resolution Minimalist Optical System Design Based on Deep Learning.

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Camera-Independent Single Image Depth Estimation from Defocus Blur

Depth Monocular Estimation with Attention-based Encoder-Decoder Network from Single Image

Deep eyes: Joint depth inference using monocular and binocular cues

End-to-end Computational Optics with a Singlet Lens for Large Depth-of-field Imaging

Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos

Depth Estimation From a Single Optical Encoded Image Using a Learned Colored-Coded Aperture

Towards Single-Lens Controllable Depth-of-Field Imaging via All-in-Focus Aberration Correction and Monocular Depth Estimation

Depth from Coupled Optical Differentiation

Depth Is All You Need for Monocular 3D Detection

Large Depth-of-field Computational Imaging with Multi-Spectral and Dual-Aperture Optics.

NDDepth: Normal-Distance Assisted Monocular Depth Estimation

DCDepth: Progressive Monocular Depth Estimation in Discrete Cosine Domain

Monocular Depth Estimation Based on Dilated Convolutions and Feature Fusion

Ranking-Based Salient Object Detection and Depth Prediction for Shallow Depth-of-Field

Out-of-focus: Learning Depth from Image Bokeh for Robotic Perception