A Double Feature Fusion Network with Progressive Learning for Sharper Inpainting

Zhichao Zhang,Hui Chen,Jinsheng Deng,Xiaoqing Yin

DOI: https://doi.org/10.1109/ijcnn52387.2021.9534018

2021-01-01

Abstract:Deep learning methods in computer vision have shown tremendous progress recently. Most image restoration methods are based on single feature extraction. However, image restoration requires various types of information to reconstruct the properties such as structure, color, texture details. Low-quality pictures are widespread in the real world because of the natural conditions and unstable photography. Therefore, we propose a double feature fusion network (DFFNet) to overcome this challenge in image inpainting. DFFNet extracts deblurring and structure features simultaneously and adopts multipath refinement framework aided by multipath contextual attention modules to restore the mask region in a coarse-to-fine manner. Thus, our work can restore blurred and large masked images into sharp and complete images and neglect the meaningless occlusions. It outperforms the state-of-the-art combined methods for sharper and smarter inpainting. First, we design a multi-path refinement network that can extract multi-features for further refinement and coarse inpainting. Second, multipath contextual attention modules are reconstructed for receiving edge and deblur features and finer inpainting results. Third, multi-stage synthesis loss function and double feature fusion units ensure that the restored images have the same structure and texture similarity as the original ground-truth counterparts. Therefore, the progressive learning framework DFFNet adopts key elements of image restoration information such as structure and texture, and refines the background for removing meaningless noise for sharper and smarter inpainting.