Multi-Task Affinity Propagation Based Natural Image Matting

Renkai Zhang,Nong Sang

DOI: https://doi.org/10.1109/icip51287.2024.10647469

2024-01-01

Abstract:Image matting, aiming to accurately extract foreground objects by estimating their opacity against the background, has made remarkable progress through deep-learning approaches. Nevertheless, the majority of these methods require a user-defined auxiliary input, such as a trimap, which limits their applications in real-world scenarios. There are many auxiliary input-free methods that have been proposed by now, and some of them adopt a multi-task learning framework that includes a shared encoder and two separate decoders. However, these methods lack interactions between the two decoders, or interactions are implemented through simple summation or concatenation. Unfortunately, the integration of different features may cause negative transfer and limit the model performance due to the invisible information transmission process. To address the issue, we introduce the Pattern-Affinitive Propagation Module (PAP) to explicitly model cross-task propagation and task-specific propagation. Furthermore, image matting not only requires high-resolution detail features, but also semantic features. However, current CNN-based methods have limited receptive fields, making it challenging to capture global semantic features. Therefore, we design a module that integrates Dilated Convolution and Spectral Transformer (DSM), which can effectively capture global features and enhance global-local feature fusion. Extensive experiments on AM-2k and P3M-10k datasets demonstrate the superiority of our method.