Abstract:The incomplete construction of optical image time series caused by cloud contamination is one of the major limitations facing the application of optical satellite images in crop monitoring. Thus, the construction of a complete optical image time series via image reconstruction of cloud-contaminated regions is essential for thematic mapping in croplands. This study investigates the potential of multi-temporal conditional generative adversarial networks (MTcGANs) that use a single synthetic aperture radar (SAR) image acquired on a prediction date and a pair of SAR and optical images acquired on a reference date in the context of early-stage crop monitoring. MTcGAN has an advantage over conventional SAR-to-optical image translation methods as it allows input data of various compositions. As the prediction performance of MTcGAN depends on the input data composition, the variations in the prediction performance should be assessed for different input data combination cases. Such an assessment was performed through experiments using Sentinel-1 and -2 images acquired in the US Corn Belt. MTcGAN outperformed existing SAR-to-optical image translation methods, including Pix2Pix and supervised CycleGAN (S-CycleGAN), in cases representing various input compositions. In particular, MTcGAN was substantially superior when there was little change in crop vitality between the reference and prediction dates. For the SWIR1 band, the root mean square error of MTcGAN (0.021) for corn was significantly improved by 54.4% and 50.0% compared to Pix2Pix (0.046) and S-CycleGAN (0.042), respectively. Even when there were large changes in crop vitality, the prediction accuracy of MTcGAN was more than twice that of Pix2Pix and S-CycleGAN. Without considering the temporal intervals between input image acquisition dates, MTcGAN was found to be beneficial when crops were visually distinct in both SAR and optical images. These experimental results demonstrate the potential of MTcGAN in SAR-to-optical image translation for crop monitoring during the early growth stage and can serve as a guideline for selecting appropriate input images for MTcGAN.

Synthesis of Multispectral Optical Images From SAR/Optical Multitemporal Data Using Conditional Generative Adversarial Networks

Synthesizing Optical and SAR Imagery From Land Cover Maps and Auxiliary Raster Data

Electrooptical Image Synthesis from SAR Imagery Using Generative Adversarial Networks

A SAR-to-Optical Image Translation Method Based on Conditional Generation Adversarial Network (cGAN)

TSGAN: An Optical-to-SAR Dual Conditional GAN for Optical based SAR Temporal Shifting

Atrous cGAN for SAR to Optical Image Translation

Multi-Temporal Sentinel-1 and -2 Data Fusion for Optical Image Simulation

Image translation between high-resolution optical and synthetic aperture radar (SAR) data

Spectral Image Visualization Using Generative Adversarial Networks

Multispectral Band-Aware Generation of Satellite Images across Domains Using Generative Adversarial Networks and Contrastive Learning

Data Augmentation via Mixed Class Interpolation using Cycle-Consistent Generative Adversarial Networks Applied to Cross-Domain Imagery

Multi-Scale translation method from SAR to optical remote sensing images based on conditional generative adversarial network

Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming

Conditional Generative Adversarial Networks for Data Augmentation and Adaptation in Remotely Sensed Imagery

Single Satellite Optical Imagery Dehazing Using SAR Image Prior Based on Conditional Generative Adversarial Networks.

Assessing the Potential of Multi-Temporal Conditional Generative Adversarial Networks in SAR-to-Optical Image Translation for Early-Stage Crop Monitoring

K-Means Clustering Guided Generative Adversarial Networks for SAR-Optical Image Matching

Enhancing crop classification accuracy by synthetic SAR-Optical data generation using deep learning

Synthetic Data Augmentation Using Multiscale Attention CycleGAN for Aircraft Detection in Remote Sensing Images

A Comparative Analysis of GAN-Based Methods for SAR-to-Optical Image Translation

Exploring the Potential of Unsupervised Image Synthesis for SAR-Optical Image Matching.