Quaternion Generative Adversarial Neural Networks and Applications to Color Image Inpainting

Duan Wang,Dandan Zhu,Meixiang Zhao,Zhigang Jia

2024-06-17

Abstract:Color image inpainting is a challenging task in imaging science. The existing method is based on real operation, and the red, green and blue channels of the color image are processed separately, ignoring the correlation between each channel. In order to make full use of the correlation between each channel, this paper proposes a Quaternion Generative Adversarial Neural Network (QGAN) model and related theory, and applies it to solve the problem of color image inpainting with large area missing. Firstly, the definition of quaternion deconvolution is given and the quaternion batch normalization is proposed. Secondly, the above two innovative modules are applied to generate adversarial networks to improve stability. Finally, QGAN is applied to color image inpainting and compared with other state-of-the-art algorithms. The experimental results show that QGAN has superiority in color image inpainting with large area missing.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to solve the challenges encountered in color image inpainting, especially when there are large - area missing parts in the image. Traditional image inpainting methods are usually based on real - number operations and process the red, green, and blue channels of color images separately, ignoring the correlations between the channels. This leads to poor results when dealing with large - area missing parts.

To solve this problem, the author proposes a new model and related theories based on the Quaternion Generative Adversarial Network (QGAN). By introducing innovative modules such as quaternion convolution and quaternion deconvolution, QGAN can fully utilize the correlations between the channels, thereby more effectively inpainting color images with large - area missing parts.

Specifically, the main contributions of the paper include:

1. **Define quaternion deconvolution**: Propose a new quaternion deconvolution operation for use in the generative adversarial network.
2. **Introduce quaternion batch normalization**: In order to improve the stability and expressiveness of the network, propose a quaternion batch normalization method.
3. **Apply to color image inpainting**: Apply QGAN to the color image inpainting task and conduct comparative experiments with the existing state - of - the - art algorithms.

The experimental results show that QGAN has obvious advantages in dealing with the inpainting of color images with large - area missing parts.

### Key formulas

- Quaternion representation:
\[
q = q_0+q_1i + q_2j+q_3k
\]
where \(q_0, q_1, q_2, q_3\) are real numbers, and \(i, j, k\) are three imaginary parts.

- The modulus of a quaternion:
\[
|q|=\sqrt{q_0^2 + q_1^2+q_2^2+q_3^2}
\]

- Quaternion convolution:
\[
A\ast W = [z_{i'j'}]\in Q^{(m - d + 1)\times(n - d + 1)}
\]
where
\[
z_{i'j'}=\sum_{i = 1}^{d}\sum_{j = 1}^{d}\frac{1}{s_{ij}}w_{ij}a_{(i'+i)(j'+j)}w_{ij}^*
\]

- Quaternion deconvolution:
\[
a_{ij}\circledast w_{ij}=s_{ij}R_{ij}^T\vec{a}_{ij}
\]

- Loss functions:
- Contextual Loss:
\[
L_c(z|y, M)=\|M\cdot G(z)-M\cdot y\|_1
\]
- Prior Loss:
\[
L_p(z)=\lambda\log(1 - D(G(z)))
\]

- Total loss function:
\[
L = L_c(z|y, M)+L_p(z)
\]

Through these formulas and methods, QGAN can better handle the complex problems in color image inpainting, especially in the case of large - area missing parts.