Abstract:The incomplete construction of optical image time series caused by cloud contamination is one of the major limitations facing the application of optical satellite images in crop monitoring. Thus, the construction of a complete optical image time series via image reconstruction of cloud-contaminated regions is essential for thematic mapping in croplands. This study investigates the potential of multi-temporal conditional generative adversarial networks (MTcGANs) that use a single synthetic aperture radar (SAR) image acquired on a prediction date and a pair of SAR and optical images acquired on a reference date in the context of early-stage crop monitoring. MTcGAN has an advantage over conventional SAR-to-optical image translation methods as it allows input data of various compositions. As the prediction performance of MTcGAN depends on the input data composition, the variations in the prediction performance should be assessed for different input data combination cases. Such an assessment was performed through experiments using Sentinel-1 and -2 images acquired in the US Corn Belt. MTcGAN outperformed existing SAR-to-optical image translation methods, including Pix2Pix and supervised CycleGAN (S-CycleGAN), in cases representing various input compositions. In particular, MTcGAN was substantially superior when there was little change in crop vitality between the reference and prediction dates. For the SWIR1 band, the root mean square error of MTcGAN (0.021) for corn was significantly improved by 54.4% and 50.0% compared to Pix2Pix (0.046) and S-CycleGAN (0.042), respectively. Even when there were large changes in crop vitality, the prediction accuracy of MTcGAN was more than twice that of Pix2Pix and S-CycleGAN. Without considering the temporal intervals between input image acquisition dates, MTcGAN was found to be beneficial when crops were visually distinct in both SAR and optical images. These experimental results demonstrate the potential of MTcGAN in SAR-to-optical image translation for crop monitoring during the early growth stage and can serve as a guideline for selecting appropriate input images for MTcGAN.

Multi-Scale translation method from SAR to optical remote sensing images based on conditional generative adversarial network

A SAR-to-Optical Image Translation Method Based on Conditional Generation Adversarial Network (cGAN)

Image translation between high-resolution optical and synthetic aperture radar (SAR) data

Serial GANs: A Feature-Preserving Heterogeneous Remote Sensing Image Transformation Model.

Domain transfer net based on U-Net and transformer for synthetic aperture radar-to-optical image translation

A Dual-Generator Translation Network Fusing Texture and Structure Features for SAR and Optical Image Matching

Reciprocal translation between SAR and optical remote sensing images with cascaded-residual adversarial networks

High-Resolution SAR-to-Multispectral Image Translation Based on S2MS-GAN

A Comparative Analysis of GAN-Based Methods for SAR-to-Optical Image Translation

ADD-UNet: an Adjacent Dual-Decoder UNet for SAR-to-Optical Translation.

An unpaired SAR-to-optical image translation method based on Schrödinger bridge network and multi-scale feature fusion

Translating SAR to Optical Images for Assisted Interpretation

Dialectical GAN for SAR Image Translation: From Sentinel-1 to TerraSAR-X

Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task

Conditional Diffusion for SAR to Optical Image Translation

TSGAN: An Optical-to-SAR Dual Conditional GAN for Optical based SAR Temporal Shifting

Assessing the Potential of Multi-Temporal Conditional Generative Adversarial Networks in SAR-to-Optical Image Translation for Early-Stage Crop Monitoring

Multi-Scale Convolutional Neural Network for SAR Image Semantic Segmentation

Accelerating Diffusion for SAR-to-Optical Image Translation via Adversarial Consistency Distillation

A-CGAN based transformation from ISAR to optical image

Synthesis of Multispectral Optical Images From SAR/Optical Multitemporal Data Using Conditional Generative Adversarial Networks