Adaptive Information Fusion Network for Arbitrary Style Transfer
Jiaoju Zhou,Feng Gao,Xuebo Yang,Weiyang Lin
DOI: https://doi.org/10.1109/IECON51785.2023.10312640
2023-01-01
Abstract:Style transfer techniques have found wide-ranging applications in diverse domains, including image enhancement, film and animation production, augmented reality, and social media, garnering significant attention across various disciplines. However, prevailing approaches mainly rely on data-driven and adaptive normalization techniques for learning transformation matrices, while overlooking critical aspects such as multi-scale feature extraction, local-global distributions, and spatial-channel dimensions information. In this paper, we present a novel Adaptive Information Fusion Network (AIFN), comprising an encoder, an information fusion module, and a decoder symmetric to the encoder. Specifically, the information fusion module receives multi-scale feature maps extracted from a pre-trained encoder, consisting of three parallel sub-modules: Adaptive Attention Normalization (AdaAttN), Spatial-channel correlation, and a Linear submodule. Through adaptive learning of sub-module weights, we seamlessly integrate style features to achieve a harmonious fusion. Furthermore, we introduce illumination loss, ink wash loss, and identity loss to enhance stylization performance concerning lighting variations, global hue and diffusion mode, while retaining accurate and rich content features. Through comparison and ablation experiments, the proposed method produces high-quality stylized images, demonstrating excellent performance in arbitrary style transfer tasks.