Thermal-NeRF: Neural Radiance Fields from an Infrared Camera

Tianxiang Ye,Qi Wu,Junyuan Deng,Guoqing Liu,Liu Liu,Songpengcheng Xia,Liang Pang,Wenxian Yu,Ling Pei

2024-03-15

Abstract:In recent years, Neural Radiance Fields (NeRFs) have demonstrated significant potential in encoding highly-detailed 3D geometry and environmental appearance, positioning themselves as a promising alternative to traditional explicit representation for 3D scene reconstruction. However, the predominant reliance on RGB imaging presupposes ideal lighting conditions: a premise frequently unmet in robotic applications plagued by poor lighting or visual obstructions. This limitation overlooks the capabilities of infrared (IR) cameras, which excel in low-light detection and present a robust alternative under such adverse scenarios. To tackle these issues, we introduce Thermal-NeRF, the first method that estimates a volumetric scene representation in the form of a NeRF solely from IR imaging. By leveraging a thermal mapping and structural thermal constraint derived from the thermal characteristics of IR imaging, our method showcasing unparalleled proficiency in recovering NeRFs in visually degraded scenes where RGB-based methods fall short. We conduct extensive experiments to demonstrate that Thermal-NeRF can achieve superior quality compared to existing methods. Furthermore, we contribute a dataset for IR-based NeRF applications, paving the way for future research in IR NeRF reconstruction.

Computer Vision and Pattern Recognition,Robotics

What problem does this paper attempt to address?

The paper aims to address the poor performance of traditional RGB image-based NeRF (Neural Radiance Fields) methods in adverse environments such as low light and smoke. Specifically:

1. **Proposing Thermal-NeRF**: The paper introduces a new method called Thermal-NeRF, which is the first attempt to estimate neural radiance fields using only infrared (IR) camera data. This method can reconstruct high-quality 3D scenes in visually degraded environments.

2. **Thermal Mapping and Structural Constraints**: The authors propose a thermal mapping technique to model thermal values in IR images and introduce structural thermal constraints to leverage structural information in the images. These innovations help address the issues of sparse features and limited textures in IR images.

3. **Experimental Validation and Dataset Contribution**: Extensive experiments validate the superior performance of Thermal-NeRF under various adverse conditions, and the authors contribute a dataset specifically for IR-based NeRF applications, filling a gap in existing datasets.

In summary, this research aims to enhance 3D scene reconstruction capabilities in complex environments by developing a novel IR imaging-based NeRF method, especially under conditions where traditional RGB cameras struggle to perform.