Polarimetric Inverse Rendering for Transparent Shapes Reconstruction

Mingqi Shao,Chongkun Xia,Dongxu Duan,Xueqian Wang

DOI: https://doi.org/10.1109/tmm.2024.3371792

IF: 7.3

2024-01-01

IEEE Transactions on Multimedia

Abstract:The acquisition of transparent 3D shapes will facilitate many multimedia and computer vision tasks, such as game/movie production and virtual enrioment applications. In this work, we propose a novel method for detailed reconstruction of transparent objects by exploiting polarimetric cues. Most of existing transparent shapes reconstruction methods usually lack sufficient constraints and suffer from the over-smooth problem. Hence, we introduce polarization information as a complementary cue. Specifically, we employ the implicit representation for object's geometry with a neural network, while the polarization render is capable of differentiably rendering the object's polarization images from given illumination configuration. However, direct comparison of rendered polarization images to the real-world captured images will have additional errors due to the transmission in the transparent object. To make the polarimetric cues technically feasible on transparent shapes reconstruction, the concept of reflection percentage which represents proportion of the reflection component is introduced as the weight of the polarization loss. Based on controllable environment setup, we build a polarization dataset containing several solid and smooth transparent objects to verify our method. Experimental results show that our method is capable of recovering detailed shapes and improving reconstruction quality of transparent objects.

computer science, information systems,telecommunications, software engineering