Synthesis and evaluation of seamless, large-scale, multispectral satellite images using Generative Adversarial Networks on land use and land cover and Sentinel-2 data

Torben Dedring Andreas Rienow Interdisciplinary Geographic Information Sciences,Geomatics Research Group,Institute of Geography,Faculty of Geosciences,Ruhr University Bochum,Bochum,Germany
DOI: https://doi.org/10.1080/15481603.2024.2364460
IF: 6.397
2024-06-11
GIScience & Remote Sensing
Abstract:Artificial intelligence (AI) began to make its way into geoinformation science several decades ago and since then has constantly brought forth new cutting-edge approaches for diverse geographic use cases. AI and deep learning methods have become essential approaches for land use and land cover (LULC) classifications, which are important in urban planning and regional management. While the extraction of LULC information from multispectral satellite images has been a well-studied part of past and present research, only a few studies emerged about the recovery of spectral properties from LULC information. Estimates of the spectral characteristics of LULC categories could enrich LULC forecasting models by providing necessary information to delineate vegetation indices or microclimatic parameters. We train two identical Conditional Generative Adversarial Networks (CGAN) to synthesize a multispectral Sentinel-2 image based on different combinations of open-source LULC data sets. Large-scale synthetic multispectral satellite images of the administrative region of Bonn and Rhein-Sieg in Germany are generated with a Euclidean distance-based patch-fusion method. The approach generated a realistic-looking satellite image without noticeable seams between patch borders. Based on several metrics, such as difference calculations, the spectral information divergence (SID), and the Fréchet inception distance (FID), we evaluate the resulting images. The models reach mean SIDs as low as 0.026 for urban fabrics and forests and FIDs below 90 for bands B2 and B5 showing that the CGAN is capable of synthesizing distinct synthetic features matching with features typical for respective LULC categories and manages to mimic multispectral signatures. The method used in this paper to generate large-scale synthetic multispectral satellite images can be used as an approach to support scenario-oriented sustainable urban planning.
geography, physical,remote sensing
What problem does this paper attempt to address?
### What problems does this paper attempt to solve? This paper aims to explore how to use Conditional Generative Adversarial Networks (CGAN) to synthesize seamless, large - scale multispectral satellite images from open - source Land Use and Land Cover (LULC) data. Specifically, the research mainly focuses on the following aspects: 1. **Conversion of LULC information to multispectral images**: - Investigate whether open - source LULC data can be used as input to generate images with features corresponding to the underlying LULC categories through CGAN. 2. **Reliable synthesis of multispectral properties**: - Explore whether CGAN can reliably synthesize multispectral characteristics and ensure that the spectral characteristics of the generated images in different bands are similar to those of real images. 3. **Performance differences in different LULC categories**: - Analyze whether there are differences in the performance of CGAN in different LULC categories, such as urban structures, forests, etc. 4. **Performance differences in different image channels**: - Examine whether there are significant differences in the performance of CGAN in different image channels (such as B2 and B5 bands). ### Method overview To achieve the above goals, the authors adopted the following methods: - **Data preparation**: Use open - source LULC datasets from different federal states in western Germany as training data, and select data from Bonn and Rhein - Sieg regions for testing. - **Model architecture**: Adjust based on the Pix2Pix network architecture, including a generator with a U - NET structure and a discriminator with a Patch GAN structure. - **Post - processing**: Adopt a patch - fusion method based on Euclidean distance to generate seamless large - scale multispectral satellite images. - **Evaluation metrics**: Use multiple evaluation metrics, including Root Mean Square Error (RMSE), Spectral Information Divergence (SID), Fréchet Inception Distance (FID), etc., to evaluate the quality of the generated images. ### Conclusion Through experiments, the authors have proven that CGAN can successfully synthesize realistic multispectral satellite images from LULC data to a certain extent, especially performing well in some specific LULC categories and image channels. This provides new possibilities for future applications in urban planning and regional management.