Fast Satellite Tensorial Radiance Field for Multi-date Satellite Imagery of Large Size

Tongtong Zhang,Yuanxiang Li

2023-09-21

Abstract:Existing NeRF models for satellite images suffer from slow speeds, mandatory solar information as input, and limitations in handling large satellite images. In response, we present SatensoRF, which significantly accelerates the entire process while employing fewer parameters for satellite imagery of large size. Besides, we observed that the prevalent assumption of Lambertian surfaces in neural radiance fields falls short for vegetative and aquatic elements. In contrast to the traditional hierarchical MLP-based scene representation, we have chosen a multiscale tensor decomposition approach for color, volume density, and auxiliary variables to model the lightfield with specular color. Additionally, to rectify inconsistencies in multi-date imagery, we incorporate total variation loss to restore the density tensor field and treat the problem as a denosing <a class="link-external link-http" href="http://task.To" rel="external noopener nofollow">this http URL</a> validate our approach, we conducted assessments of SatensoRF using subsets from the spacenet multi-view dataset, which includes both multi-date and single-date multi-view RGB images. Our results clearly demonstrate that SatensoRF surpasses the state-of-the-art Sat-NeRF series in terms of novel view synthesis performance. Significantly, SatensoRF requires fewer parameters for training, resulting in faster training and inference speeds and reduced computational demands.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are several key issues existing in the existing Neural Radiance Field (NeRF) models when processing satellite images:

1. **Slow speed**: The existing NeRF models are slow when processing satellite images, especially when dealing with large - scale satellite images.
2. **Dependence on solar information**: The existing NeRF models require additional solar direction information as input, which may not be available in some datasets and increases the computational burden.
3. **Limited ability to process large - scale images**: The existing NeRF models have limitations when processing large - scale satellite images, especially in the trade - off between memory consumption and batch size.
4. **Over - simplified light - field assumption**: The existing NeRF models assume that the earth's surface follows the strict Lambertian law, but the actual surfaces (such as vegetation, water bodies, asphalt pavements) often have bidirectional reflection characteristics, which leads to poor performance of the model in these areas.

To solve these problems, the paper proposes the SatensoRF model, which has the following features:

- **Acceleration and reduction of the number of parameters**: Through the multi - scale tensor decomposition method, SatensoRF significantly accelerates the whole process and uses fewer parameters to process large - scale satellite images.
- **No need for additional solar information**: SatensoRF eliminates the need for solar direction information, making it more flexible when dealing with different datasets.
- **Improved light - field model**: A light - field model considering BRDF characteristics is introduced, which can better capture the details of non - Lambertian surfaces such as vegetation and water bodies.
- **Processing of transient objects**: Through low - rank tensor representation and total variation denoising techniques, SatensoRF can effectively handle the influence of transient objects (such as vehicles, pedestrians).

Through these improvements, SatensoRF outperforms the existing Sat - NeRF series models in the new view synthesis performance of multi - date satellite images, and performs excellently in terms of training and inference speed and computational requirements.