WildLight: In-the-wild Inverse Rendering with a Flashlight

Ziang Cheng,Junxuan Li,Hongdong Li

2023-03-25

Abstract:This paper proposes a practical photometric solution for the challenging problem of in-the-wild inverse rendering under unknown ambient lighting. Our system recovers scene geometry and reflectance using only multi-view images captured by a smartphone. The key idea is to exploit smartphone's built-in flashlight as a minimally controlled light source, and decompose image intensities into two photometric components -- a static appearance corresponds to ambient flux, plus a dynamic reflection induced by the moving flashlight. Our method does not require flash/non-flash images to be captured in pairs. Building on the success of neural light fields, we use an off-the-shelf method to capture the ambient reflections, while the flashlight component enables physically accurate photometric constraints to decouple reflectance and illumination. Compared to existing inverse rendering methods, our setup is applicable to non-darkroom environments yet sidesteps the inherent difficulties of explicit solving ambient reflections. We demonstrate by extensive experiments that our method is easy to implement, casual to set up, and consistently outperforms existing in-the-wild inverse rendering techniques. Finally, our neural reconstruction can be easily exported to PBR textured triangle mesh ready for industrial renderers.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the problem of outdoor inverse rendering under unknown environmental lighting conditions. Specifically, the authors propose a practical photometric solution that can recover the geometry and reflectance of outdoor scenes under unknown environmental lighting. Compared to existing inverse rendering methods, this approach is not only suitable for non-darkroom environments but also avoids the inherent problems associated with explicitly solving for environmental reflections.

The core idea of the paper is to use the built-in flashlight of a smartphone as a controlled light source and decompose the image intensity into two parts: one corresponding to the static appearance (i.e., environmental light flux) and the other to the dynamic reflection caused by the moving flashlight. This method does not require paired acquisition of flash and non-flash images but instead relies on neural radiance field techniques to capture environmental reflections and provides physically accurate photometric constraints through the flashlight component, thereby separating reflectance and illumination.

Experimental results show that this method is easy to implement, simple to set up, and outperforms existing outdoor inverse rendering techniques in various experiments. Additionally, the reconstructed objects can be easily converted into standard formats suitable for industrial renderers (such as PBR texture triangle meshes).