Learning Structure-Aware Transformations for Arbitrary Image Style Transfer

Pan Li,Lei Zhao,Duanqing Xu,Dongming Lu
DOI: https://doi.org/10.1117/12.2623448
2022-01-01
Abstract:Arbitrary image style transfer aims to mimic the artistic features of a randomly given style image while maintaining the semantic characters of a reference content image. Existing algorithms have achieved astonishing style transfer results. However, they are insufficient to capture the global information of the content image due to the locality in convolutional neural networks. As a result, the content structures of the stylized images are disrupted by scattered textures. To address this issue, we propose to embed additional global spatial information of the reference content image into image style transfer algorithms with the adaptive instance normalization trick. We train a light-weighted network to distill the global structural information provided by the depth map of the reference content image. We then utilize the refined results to enhance the transformations between deep features of the inputs. Experimental results show that our proposed method can generate stylized outputs with more consistent structures and less unwanted textures, achieving impressive visual effects.
What problem does this paper attempt to address?