Abstract:Image editing and compositing have become ubiquitous in entertainment, from digital art to AR and VR experiences. To produce beautiful composites, the camera needs to be geometrically calibrated, which can be tedious and requires a physical calibration target. In place of the traditional multi-image calibration process, we propose to infer the camera calibration parameters such as pitch, roll, field of view, and lens distortion directly from a single image using a deep convolutional neural network. We train this network using automatically generated samples from a large-scale panorama dataset, yielding competitive accuracy in terms of standard `2 error. However, we argue that minimizing such standard error metrics might not be optimal for many applications. In this work, we investigate human sensitivity to inaccuracies in geometric camera calibration. To this end, we conduct a large-scale human perception study where we ask participants to judge the realism of 3D objects composited with correct and biased camera calibration parameters. Based on this study, we develop a new perceptual measure for camera calibration and demonstrate that our deep calibration network outperforms previous single-image based calibration methods both on standard metrics as well as on this novel perceptual measure. Finally, we demonstrate the use of our calibration network for several applications, including virtual object insertion, image retrieval, and compositing. A demonstration of our approach is available at <a class="link-external link-https" href="https://lvsn.github.io/deepcalib" rel="external noopener nofollow">this https URL</a> .

Deep-learning-based Adaptive Camera Calibration for Various Defocusing Degrees

Calibration-free Deep Optics for Depth Estimation with Precise Simulation

High-accuracy Calibration of Cameras Without Depth of Field and Target Size Limitations.

Feature points extraction of defocused images using deep learning for camera calibration

A Robust and Accurate Calibration Method for Out-of-focus Camera.

A Novel Technique of Image-Based Camera Calibration in Depth-from-Defocus

Multi-channel Residual Network Model for Accurate Estimation of Spatially-Varying and Depth-Dependent Defocus Kernels.

Deep Learning for Camera Calibration and Beyond: A Survey

A Deep-Learning Based High-Accuracy Camera Calibration Method for Large-Scale Scene

Deep Learning for Camera Autofocus

Feature Extraction Reimagined: Achieving Superior Accuracy in Camera Calibration

Learning to Deblur using Light Field Generated and Real Defocus Images

Camera Geometric Calibration Using Dynamic Single-Pixel Illumination With Deep Learning Networks

Camera Calibration Approach Based On Adaptive Active Target

Learning-Based Framework for Camera Calibration with Distortion Correction and High Precision Feature Detection

A Deep Perceptual Measure for Lens and Camera Calibration

Learning-Based Distortion Correction and Feature Detection for High Precision and Robust Camera Calibration

Effective Improvement for Depth Estimated Based on Defocus Images

Adaptive Adjustment of Brightness and Blur of the Camera for High Precision Internal Parameter Calibration

An Effective Method for Camera Calibration in Defocus Scene with Circular Gratings

Defocus Deblurring: a Designed Deep Model Based on CNN