Abstract:Object transfiguration is a subtask of the image-to-image translation, which translates two independent image sets and has a wide range of applications. Recently, some studies based on Generative Adversarial Network (GAN) have achieved impressive results in the image-to-image translation. However, the object transfiguration task only translates regions containing target objects instead of whole images; most of the existing methods never consider this issue, which results in mistranslation on the backgrounds of images. To address this problem, we present a novel pipeline called Deep Attention Unit Generative Adversarial Networks (DAU-GAN). During the translating process, the DAU computes attention masks that point out where the target objects are. DAU makes GAN concentrate on translating target objects while ignoring meaningless backgrounds. Additionally, we construct an attention-consistent loss and a background-consistent loss to compel our model to translate intently target objects and preserve backgrounds further effectively. We have comparison experiments on three popular related datasets, demonstrating that the DAU-GAN achieves superior performance to the state-of-the-art. We also export attention masks in different stages to confirm its effect during the object transfiguration task. The proposed DAU-GAN can translate object effectively as well as preserve backgrounds information at the same time. In our model, DAU learns to focus on the most important information by producing attention masks. These masks compel DAU-GAN to effectively distinguish target objects and backgrounds during the translation process and to achieve impressive translation results in two subsets of ImageNet and CelebA. Moreover, the results show that we cannot only investigate the model from the image itself but also research from other modal information.

IIT-GAT: Instance-level Image Transformation Via Unsupervised Generative Attention Networks with Disentangled Representations

Unsupervised Image-to-Image Translation with Generative Adversarial Networks.

Unpaired Salient Object Translation Via Spatial Attention Prior

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

AttentionGAN: Unpaired Image-to-Image Translation Using Attention-Guided Generative Adversarial Networks

CSAGAN: Channel and Spatial Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

Image-to-image Translation Model Combining GAN and Multi-angle Attention

AT-GAN - Attention Transfer GAN for Image-to-Image Translation.

Unsupervised Object Transfiguration with Attention

OSAGGAN: one-shot unsupervised image-to-image translation using attention-guided generative adversarial networks

DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)

A one-to-many conditional generative adversarial network framework for multiple image-to-image translations

Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network

UGC: Unified GAN Compression for Efficient Image-to-Image Translation

Unsupervised Transformation Network Based on GANs for Target-Domain Oriented Image Translation.

Attention-Based Spatial Guidance for Image-to-Image Translation

Image Translation with Attention Mechanism Based on Generative Adversarial Networks.

Unsupervised image-to-image translation with multiscale attention generative adversarial network

Towards Unsupervised Deformable-Instances Image-to-Image Translation

Unsupervised Image-to-Image Translation with Generative Prior

Image Translation with Dual‐directional Generative Adversarial Networks