Abstract:Satellite image generation has a wide range of applications. For example, parts of images must be restored in areas obscured by clouds or cloud shadows or areas that must be anonymized. The need to cover a large area with the generated images faces the challenge that separately generated images must maintain the structural and color continuity between the adjacent generated images as well as the actual ones. This study presents a modified architecture of the generative adversarial network (GAN) pix2pix that ensures the integrity of the generated remote sensing images. The pix2pix model comprises a U-Net generator and a PatchGAN discriminator. The generator was modified by expanding the input set with images representing the known parts of ground truth and the respective mask. Data used for the generative model consist of Sentinel-2 (S2) RGB satellite imagery as the target data and OpenStreetMap mapping data as the input. Since forested areas and fields dominate in images, a Kneedle clusterization method was applied to create datasets that better represent the other classes, such as buildings and roads. The original and updated models were trained on different datasets and their results were evaluated using gradient magnitude (GM), Fréchet inception distance (FID), structural similarity index measure (SSIM), and multiscale structural similarity index measure (MS-SSIM) metrics. The models with the updated architecture show improvement in gradient magnitude, SSIM, and MS-SSIM values for all datasets. The average GMs of the junction region and the full image are similar (do not exceed 7%) for the images generated using the modified architecture whereas it is more than 13% higher in the junction area for the images generated using the original architecture. The importance of class balancing is demonstrated by the fact that, for both architectures, models trained on the dataset with a higher ratio of classes representing buildings and roads compared to the models trained on the dataset without clusterization have more than 10% lower FID (162.673 to 190.036 for pix2pix and 173.408 to 195.621 for the modified architecture) and more than 5% higher SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) and MS-SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) values.

High Resolution Solar Image Generation Using Generative Adversarial Networks

Generation of He i 1083 nm Images from SDO AIA Images by Deep Learning

Improving the Spatial Resolution of Solar Images Using Generative Adversarial Network and Self-attention Mechanism

Improving the Spatial Resolution of Solar Images Using Super-Resolution Diffusion Generative Adversarial Networks

Super-resolution of Solar Magnetograms Using Deep Learning

A Transfer Learning Method to Generate Synthetic Synoptic Magnetograms

Solar synthetic imaging: Introducing denoising diffusion probabilistic models on SDO/AIA data

Generate Radioheliograph Image from SDO/AIA Data with Machine Learning Method

Super-resolution of SDO/HMI Magnetograms Using Novel Deep Learning Methods

Generating a radioheliograph image from SDO/AIA data with the machine learning method

A Deep Learning Approach to Generating Photospheric Vector Magnetograms of Solar Active Regions for SOHO/MDI Using SDO/HMI and BBSO Data

Hyperspectral Imagery Spatial Super-Resolution Using Generative Adversarial Network.

Algorithm Development of Cloud Removal from Solar Images Based on Pix2Pix Network

Solar Image Deconvolution by Generative Adversarial Network

Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks

Enhancing SDO/HMI images using deep learning

Data Matching of Solar Images Super-Resolution Based on Deep Learning

Near-real-time 3D Reconstruction of the Solar Coronal Parameters Based on the Magnetohydrodynamic Algorithm outside a Sphere Using Deep Learning

Magnetogram-to-Magnetogram: Generative Forecasting of Solar Evolution

Cross-Survey Image Transformation: Enhancing SDSS and DECaLS Images to Near-HSC Quality for Advanced Astronomical Analysis