Abstract:While conventional cameras offer versatility for applications ranging from amateur photography to autonomous driving, computational cameras allow for domain-specific adaption. Cameras with co-designed optics and image processing algorithms enable high-dynamic-range image recovery, depth estimation, and hyperspectral imaging through optically encoding scene information that is otherwise undetected by conventional cameras. However, this optical encoding creates a challenging inverse reconstruction problem for conventional image recovery, and often lowers the overall photographic quality. Thus computational cameras with domain-specific optics have only been adopted in a few specialized applications where the captured information cannot be acquired in other ways. In this work, we investigate a method that combines two optical systems into one to tackle this challenge. We split the aperture of a conventional camera into two halves: one which applies an application-specific modulation to the incident light via a diffractive optical element to produce a coded image capture, and one which applies no modulation to produce a conventional image capture. Co-designing the phase modulation of the split aperture with a dual-pixel sensor allows us to simultaneously capture these coded and uncoded images without increasing physical or computational footprint. With an uncoded conventional image alongside the optically coded image in hand, we investigate image reconstruction methods that are conditioned on the conventional image, making it possible to eliminate artifacts and compute costs that existing methods struggle with. We assess the proposed method with 2-in-1 cameras for optical high-dynamic-range reconstruction, monocular depth estimation, and hyperspectral imaging, comparing favorably to all tested methods in all applications.

Spliced imaging system based on axis shifting principle

Axial Adjustable Common-Path Interferometric Phase Imaging System Based on Prism Reuse

Multi-image-distance Imaging System for Extending Depth-of-field

Optical butting system

Compositing Images For An Optical Butting Camera

An Aerial Composite Imaging Method with Multiple Upright Cameras Based on Axis-Shift Theory

Transverse wide-visual field tridimensional display method and system based on spliced light field

Joint Focusing Method Based on Optimal Image-Distance Interval for Multi-Image-distance Imaging System

Multishot Hyperspectral Imaging Based on Tunable Diffractive Lenses

Research of real-time wide field image merging based on multi-cameras

Stereoscopic three-dimensional display based on multi-screen splicing

Computational Diffraction-Limited Imaging with a Non-Imaging Spatial Information Transfer Lens

Dynamically Tunable Hemispherical Electronic Eye Camera System with Adjustable Zoom Capability

Image generation system and method for stereoscopic 3D display

P‐6: A Novel 3D Display Based on Spliced Images

Line-preserving video stitching for asymmetric cameras

Split-Aperture 2-in-1 Computational Cameras

Multi-Camera System: Imaging Enhancement and Application

Lensless coherent diffraction imaging based on spatial light modulator with unknown modulation curve

Orthoscopic integral imaging for different lenslet arrays in real and virtual fields

SC-AOF: A Sliding Camera and Asymmetric Optical-Flow-Based Blending Method for Image Stitching