Abstract:In contrast to the well-investigated field of SAR-to-Optical translation, this study explores the lesser-investigated domain of Optical-to-SAR translation, a challenging field due to the ill-posed nature of this translation. The complexity arises as a single optical data can have multiple SAR representations based on the SAR viewing geometry. We propose a novel approach, termed SAR Temporal Shifting, which inputs an optical data from the desired timestamp along with a SAR data from a different temporal point but with a consistent viewing geometry as the expected SAR data, both complemented with a change map of optical data during the intervening period. This model modifies the SAR data based on the changes observed in optical data to generate the SAR data for the desired timestamp. Our model, a dual conditional Generative Adversarial Network (GAN), named Temporal Shifting GAN (TSGAN), incorporates a siamese encoder in both the Generator and the Discriminator. To prevent the model from overfitting on the input SAR data, we employed a change weighted loss function. Our approach surpasses traditional translation methods by eliminating the GAN's fiction phenomenon, particularly in unchanged regions, resulting in higher SSIM and PSNR in these areas. Additionally, modifications to the Pix2Pix architecture and the inclusion of attention mechanisms have enhanced the model's performance on all regions of the data. This research paves the way for leveraging legacy optical datasets, the most abundant and longstanding source of Earth imagery data, extending their use to SAR domains and temporal analyses. To foster further research, we provide the code, datasets used in our study, and a framework for generating paired SAR-Optical datasets for new regions of interest. These resources are available on

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper primarily explores the problem of converting optical images to synthetic aperture radar (SAR) images (Opt2SAR), a relatively under-researched and challenging field. Compared to the conversion of SAR to optical images (SAR2Opt), the Opt2SAR problem is more complex because a single optical data can have multiple SAR representations, depending on the observation geometry of the SAR. The paper proposes a novel method called "SAR Temporal Shifting," which generates SAR data for a target timestamp by inputting optical data from a specific timestamp and SAR data from another timestamp, along with a change map of the optical data between these two timestamps. This method modifies the SAR data to reflect changes in the optical data, thereby generating new SAR data. Specifically, the main contributions of the paper include: 1. **Introducing a new attention mechanism dual-encoder UNet architecture GAN model** (Temporal Shifting GAN, TSGAN), which can handle optical and SAR data and adjust SAR data through the change map to fit the new timestamp. 2. **Designing an automatic pairing Sentinel-1 and Sentinel-2 dataset download framework**, which can select the most suitable Sentinel-1 SAR orbit based on specified criteria, obtain cloud-free Sentinel-2 optical data, pair it with despeckled SAR data, and segment it into data blocks. 3. **Adopting a change-weighted L1 loss function** to prevent the model from overfitting the input SAR data in unchanged areas by assigning higher weights to the changed areas, thereby improving the model's performance. 4. **Introducing change-weighted evaluation metrics**, which can distinguish the model's performance in changed and unchanged areas, thus more finely evaluating the model's effectiveness. Through these innovations, the paper aims to overcome the "hallucination" phenomenon in the Opt2SAR task (i.e., the model generating unrealistic outputs when lacking sufficient information) and improve the model's performance in different areas, especially the high fidelity in unchanged areas. Additionally, the paper provides code, datasets, and a framework for generating new region-paired SAR-optical datasets to facilitate further research.

TSGAN: An Optical-to-SAR Dual Conditional GAN for Optical based SAR Temporal Shifting

A SAR-to-Optical Image Translation Method Based on Conditional Generation Adversarial Network (cGAN)

A Comparative Analysis of GAN-Based Methods for SAR-to-Optical Image Translation

FG-GAN: A Fine-Grained Generative Adversarial Network for Unsupervised SAR-to-Optical Image Translation.

Improved Cgan for SAR-to-Optical Image Translation

SAR-to-Optical Image Translation With Hierarchical Latent Features

Hybrid Cgan: Coupling Global and Local Features for SAR-to-Optical Image Translation

Scene Embedded Generative Adversarial Networks for Semi-Supervised SAR-to-Optical Image Translation

Serial GANs: A Feature-Preserving Heterogeneous Remote Sensing Image Transformation Model.

SAR-to-Optical Image Translation Using Supervised Cycle-Consistent Adversarial Networks

Multiscale Generative Adversarial Network Based on Wavelet Feature Learning for SAR-to-Optical Image Translation.

Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task

SAR-to-optical Image Translation by a Variational Generative Adversarial Network

Dialectical GAN for SAR Image Translation: From Sentinel-1 to TerraSAR-X

A SAR-to-Optical Image Translation Method Based on PIX2PIX.

MS-GAN: Learn to Memorize Scene for Unpaired SAR-to-Optical Image Translation

Edge-Preserving Convolutional Generative Adversarial Networks for SAR-to-Optical Image Translation

Translating SAR to Optical Images for Assisted Interpretation

Learning SAR-to-Optical Image Translation Via Diffusion Models with Color Memory

A Dual-Generator Translation Network Fusing Texture and Structure Features for SAR and Optical Image Matching

HFGAN: A Heterogeneous Fusion Generative Adversarial Network for Sar-to-Optical Image Translation.