Boosting General Trimap-free Matting in the Real-World Image

Leo Shan Wenzhang Zhou Grace Zhao
2024-05-28
Abstract:Image matting aims to obtain an alpha matte that separates foreground objects from the background accurately. Recently, trimap-free matting has been well studied because it requires only the original image without any extra input. Such methods usually extract a rough foreground by itself to take place trimap as further guidance. However, the definition of 'foreground' lacks a unified standard and thus ambiguities arise. Besides, the extracted foreground is sometimes incomplete due to inadequate network design. Most importantly, there is not a large-scale real-world matting dataset, and current trimap-free methods trained with synthetic images suffer from large domain shift problems in practice. In this paper, we define the salient object as foreground, which is consistent with human cognition and annotations of the current matting dataset. Meanwhile, data and technologies in salient object detection can be transferred to matting in a breeze. To obtain a more accurate and complete alpha matte, we propose a network called \textbf{M}ulti-\textbf{F}eature fusion-based \textbf{C}oarse-to-fine Network \textbf{(MFC-Net)}, which fully integrates multiple features for an accurate and complete alpha matte. Furthermore, we introduce image harmony in data composition to bridge the gap between synthetic and real images. More importantly, we establish the largest general matting dataset \textbf{(Real-19k)} in the real world to date. Experiments show that our method is significantly effective on both synthetic and real-world images, and the performance in the real-world dataset is far better than existing matting-free methods. Our code and data will be released soon.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The paper primarily aims to address several key issues in the field of image matting: 1. **Unclear Foreground Definition**: Existing trimap-free methods lack a unified standard in defining the foreground, leading to misidentification during training. 2. **Incomplete Foreground Extraction**: Due to insufficient network design, sometimes the foreground object cannot be completely extracted, affecting the final alpha matte quality. 3. **Domain Differences in Datasets**: Most current trimap-free methods use synthetic images for training, performing poorly on real-world images due to significant domain gaps. To solve these problems, the authors propose the following strategies: - **Clarify Foreground Definition**: Define salient objects as the foreground and assume that the foreground in the image is the salient object. This definition aligns with human cognition and is compatible with existing dataset annotation rules. - **Improve Foreground Completeness**: Adopt a "coarse-to-fine" two-stage training strategy to ensure the completeness of the foreground. Introduce multi-module fusion of low-level appearance features, high-level semantic features, and global contextual features to further enhance foreground completeness. - **Enhance Generalization to Real-World**: Introduce image harmonization techniques to process synthetic data, making its texture and brightness closer to real images. Additionally, establish the largest real-world matting dataset to date, Real-19k, containing 19,083 diverse images, effectively evaluating the real-world performance of different networks. Through these methods, the paper proposes a new Multi-Feature Fusion Coarse-to-Fine Network (MFC-Net), which not only achieves the best results on synthetic data but also significantly outperforms existing trimap-free methods on real-world datasets.