Redefining Recon: Bridging Gaps with UAVs, 360 degree Cameras, and Neural Radiance Fields

Hartmut Surmann,Niklas Digakis,Jan-Nicklas Kremer,Julien Meine,Max Schulte,Niklas Voigt

2023-11-30

Abstract:In the realm of digital situational awareness during disaster situations, accurate digital representations, like 3D models, play an indispensable role. To ensure the safety of rescue teams, robotic platforms are often deployed to generate these models. In this paper, we introduce an innovative approach that synergizes the capabilities of compact Unmaned Arial Vehicles (UAVs), smaller than 30 cm, equipped with 360 degree cameras and the advances of Neural Radiance Fields (NeRFs). A NeRF, a specialized neural network, can deduce a 3D representation of any scene using 2D images and then synthesize it from various angles upon request. This method is especially tailored for urban environments which have experienced significant destruction, where the structural integrity of buildings is compromised to the point of barring entry-commonly observed post-earthquakes and after severe fires. We have tested our approach through recent post-fire scenario, underlining the efficacy of NeRFs even in challenging outdoor environments characterized by water, snow, varying light conditions, and reflective surfaces.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the problem of how to generate high-precision 3D models in digital situational awareness during disaster scenarios using small Unmanned Aerial Vehicles (UAVs) and Neural Radiance Fields (NeRFs) technology. Specifically, the paper focuses on the following points:

1. **Enhancing 3D modeling capabilities in disaster environments**: Traditional ground robots struggle to operate effectively in complex terrains, especially in environments with a lot of rubble and debris. The paper proposes using small UAVs (less than 30 cm) equipped with 360° panoramic cameras, combined with NeRF technology, to generate high-precision 3D models.

2. **Addressing challenges in indoor and GPS-denied environments**: Large UAVs perform poorly in indoor and GPS-denied environments because of their size and reliance on GPS signals. The paper explores how to overcome these limitations through the use of small UAVs and advanced visual positioning technologies such as visual SLAM.

3. **Optimizing data processing and scene reconstruction**: The paper introduces several different methods for processing data acquired from UAVs, including WebODM, PatchMatch-Stereo-Panorama, and NeRFs. Each of these methods has its pros and cons, but NeRFs excel in generating high-quality 3D models, especially when free viewpoint selection is required.

4. **Performance validation in practical applications**: The paper validates the effectiveness of the proposed methods through real-world cases, such as a fire scene in Essen, Germany, demonstrating the potential application of these technologies in actual rescue missions.

In summary, the paper aims to provide an efficient and accurate solution for rapid 3D modeling and situational awareness in disaster environments by integrating small UAVs, 360° panoramic cameras, and NeRF technology.