Abstract:Synthetic aperture radar (SAR) images are widely used in remote sensing. Interpreting SAR images can be challenging due to their intrinsic speckle noise and grayscale nature. To address this issue, SAR colorization has emerged as a research direction to colorize gray scale SAR images while preserving the original spatial information and radiometric information. However, this research field is still in its early stages, and many limitations can be highlighted. In this paper, we propose a full research line for supervised learning-based approaches to SAR colorization. Our approach includes a protocol for generating synthetic color SAR images, several baselines, and an effective method based on the conditional generative adversarial network (cGAN) for SAR colorization. We also propose numerical assessment metrics for the problem at hand. To our knowledge, this is the first attempt to propose a research line for SAR colorization that includes a protocol, a benchmark, and a complete performance evaluation. Our extensive tests demonstrate the effectiveness of our proposed cGAN-based network for SAR colorization. The code will be made publicly available.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the colorization of synthetic aperture radar (SAR) images. Specifically, due to its inherent speckle noise and grayscale characteristics, SAR images are very difficult to interpret. To improve this situation, researchers have proposed the research direction of SAR colorization, aiming to add colors to grayscale SAR images while preserving the original spatial and radiometric information.
However, the field of SAR colorization is still in its early stage and has many limitations. This paper proposes a complete supervised - learning - based SAR colorization research framework, including:
1. **Protocol for generating synthetic color SAR images**: Create artificial color SAR images through SAR - MS image fusion technology to provide a reference standard during the training process.
2. **Multiple baseline methods**: Ranging from simple linear regression to the recent convolutional neural network architectures, covering methods with different levels of complexity.
3. **Effective cGAN method**: A method based on conditional generative adversarial networks (cGAN) for the SAR colorization task, which significantly outperforms existing neural - network - based SAR colorization methods (such as DivColSAR).
4. **Performance evaluation metrics**: Introduce multi - dimensional similarity quality measures to numerically evaluate the quality of colorized SAR images.
These contributions together form the first framework that attempts to comprehensively study SAR colorization, including protocols, benchmarking, and complete performance evaluation, thus laying the foundation for further research in this field.
### Formula Summary
1. **cGAN Loss Function**:
\[
\min_{G} \max_{D} L_{\text{cGAN}}(G, D)=\mathbb{E}[\log(D(X, Y))]+\mathbb{E}[\log(1 - D(X, G(X)))]
\]
where \(Y\) is the true label, \(X\) is the conditional input, \(G(X)\) is the image generated by the generator, \(\mathbb{E}\) represents expectation, and \(\log\) represents the natural logarithm.
2. **NRMSE (Normalized Root Mean Square Error)**:
\[
\text{NRMSE}=\frac{1}{N} \sum_{n = 1}^{N} \frac{\text{RMSE}(n)}{\mu(n)}
\]
where \(\text{RMSE}\) represents the root mean square error, \(\mu(n)\) represents the average value of the \(n\)-th band, and \(N\) is the number of bands (3 for this problem).
3. **SAM (Spectral Angle Mapping)**:
\[
\text{SAM}(x, y)=\arccos \left(\frac{\langle x, y \rangle}{\|x\|_2\cdot\|y\|_2}\right)
\]
where \(\| \cdot \|_2\) represents the \(L_2\) norm, \(\arccos\) represents the arccosine function, and \(\langle \cdot, \cdot \rangle\) represents the dot product.
4. **Q4 (General Image Quality Index Extended for Four Bands)**:
\[
Q_4=\frac{4\cdot|\sigma_{z_1z_2}|\cdot|\mu_{z_1}|\cdot|\mu_{z_2}|}{(\sigma^2_{z_1}+\sigma^2_{z_2})(\mu^2_{z_1}+\mu^2_{z_2})}
\]
where \(z_1\) and \(z_2\) are two quaternions formed by the spectral vectors of multi - band images, \(\mu_{z_1}\) and \(\mu_{z_2}\) represent the means of \(z_1\) and \(z_2\) respectively, and \(\sigma_{z_1z_2}\) represents the covariance between \(z_1\) and \(z_2\), \(\sigma^2_{z_1}\) and...