Abstract:Satellite image generation has a wide range of applications. For example, parts of images must be restored in areas obscured by clouds or cloud shadows or areas that must be anonymized. The need to cover a large area with the generated images faces the challenge that separately generated images must maintain the structural and color continuity between the adjacent generated images as well as the actual ones. This study presents a modified architecture of the generative adversarial network (GAN) pix2pix that ensures the integrity of the generated remote sensing images. The pix2pix model comprises a U-Net generator and a PatchGAN discriminator. The generator was modified by expanding the input set with images representing the known parts of ground truth and the respective mask. Data used for the generative model consist of Sentinel-2 (S2) RGB satellite imagery as the target data and OpenStreetMap mapping data as the input. Since forested areas and fields dominate in images, a Kneedle clusterization method was applied to create datasets that better represent the other classes, such as buildings and roads. The original and updated models were trained on different datasets and their results were evaluated using gradient magnitude (GM), Fréchet inception distance (FID), structural similarity index measure (SSIM), and multiscale structural similarity index measure (MS-SSIM) metrics. The models with the updated architecture show improvement in gradient magnitude, SSIM, and MS-SSIM values for all datasets. The average GMs of the junction region and the full image are similar (do not exceed 7%) for the images generated using the modified architecture whereas it is more than 13% higher in the junction area for the images generated using the original architecture. The importance of class balancing is demonstrated by the fact that, for both architectures, models trained on the dataset with a higher ratio of classes representing buildings and roads compared to the models trained on the dataset without clusterization have more than 10% lower FID (162.673 to 190.036 for pix2pix and 173.408 to 195.621 for the modified architecture) and more than 5% higher SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) and MS-SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) values.

Enhancing Pix2Pix for Remote Sensing Image Classification

Image to Image Translation Based on Differential Image Pix2Pix Model

Mapping New Realities: Ground Truth Image Creation with Pix2Pix Image-to-Image Translation

A SAR-to-Optical Image Translation Method Based on PIX2PIX.

PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening

TRANSFER LEARNING WITH PIX2PIX GAN FOR GENERATING REALISTIC PHOTOGRAPHS FROM VIEWED SKETCH ARTS

Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks

Novel Hybrid Integrated Pix2Pix and WGAN Model with Gradient Penalty for Binary Images Denoising

Dynamic-Pix2Pix: Noise Injected cGAN for Modeling Input and Target Domain Joint Distributions with Limited Training Data

Perturbation-Seeking Generative Adversarial Networks: A Defense Framework for Remote Sensing Image Scene Classification

Remote-sensing Image Super-Resolution Using Classifier-Based Generative Adversarial Networks

cGANs for Cartoon to Real-life Images

Serial GANs: A Feature-Preserving Heterogeneous Remote Sensing Image Transformation Model.

Research on Dispersion Compensation of FD-OCT System via Pix2Pix GAN Technique

A Supervised Progressive Growing Generative Adversarial Network for Remote Sensing Image Scene Classification

Image Translation with Dual‐directional Generative Adversarial Networks

Improving GAN Training via Feature Space Shrinkage

An Optimized Training Method for GAN-Based Hyperspectral Image Classification

Conditional GAN-based Remote Sensing Target Image Generation Method

PIX-GAN: Enhance Physics-Informed Estimation Via Generative Adversarial Network

Ambient-Pix2PixGAN for Translating Medical Images from Noisy Data