Abstract:This paper proposes a novel polarization sensor structure and network architecture to obtain a high-quality RGB image and polarization information. Conventional polarization sensors can simultaneously acquire RGB images and polarization information, but the polarizers on the sensor degrade the quality of the RGB images. There is a trade-off between the quality of the RGB image and polarization information as fewer polarization pixels reduce the degradation of the RGB image but decrease the resolution of polarization information. Therefore, we propose an approach that resolves the trade-off by sparsely arranging polarization pixels on the sensor and compensating for low-resolution polarization information with higher resolution using the RGB image as a guide. Our proposed network architecture consists of an RGB image refinement network and a polarization information compensation network. We confirmed the superiority of our proposed network in compensating the differential component of polarization intensity by comparing its performance with state-of-the-art methods for similar tasks: depth completion. Furthermore, we confirmed that our approach could simultaneously acquire higher quality RGB images and polarization information than conventional polarization sensors, resolving the trade-off between the quality of RGB images and polarization information. The baseline code and newly generated real and synthetic large-scale polarization image datasets are available for further research and development.

Polarized 3D: synthesis of polarization and depth cues for enhanced 3D sensing.

Polarized 3D: High-Quality Depth Sensing with Polarization Cues

Depth Sensing Using Geometrically Constrained Polarization Normals

Polarimetric Target Depth Sensing in Ambient Illumination Based on Polarization-Coded Structured Light.

Robust Depth Enhancement via Polarization Prompt Fusion Tuning

Polarization structured light 3D depth image sensor for scenes with reflective surfaces

Target Enhanced 3D Reconstruction Based on Polarization-Coded Structured Light.

Detecting Traversable Area and Water Hazards for the Visually Impaired with a Prgb-D Sensor.

Predicting Polarization Beyond Semantics for Wearable Robotics.

Polarimetric Multi-view Stereo

DPS-Net: Deep Polarimetric Stereo Depth Estimation

SPIDeRS: Structured Polarization for Invisible Depth and Reflectance Sensing

Polarimetric Multi-View 3d Reconstruction Based On Parallax Angle And Zenith Angle Optimization

Single-shot Passive 3D and Polarization Vision Using a Monocular Metasurface Camera

Polarimetric Helmholtz Stereopsis

Shape from Polarization of low-texture objects with rough depth information

Simultaneous Acquisition of High Quality RGB Image and Polarization Information using a Sparse Polarization Sensor

Depth and Thermal Sensor Fusion to Enhance 3D Thermographic Reconstruction

Shape recovery from fusion of polarization binocular vision and shading

Polarimetric compressed sensing with hollow, self-assembled diffractive films

Polarimetric Multi-View Inverse Rendering