Abstract:Mask-guided matting networks have achieved significant improvements and have shown great potential in practical applications in recent years. However, simply learning matting representation from synthetic and lack-of-real-world-diversity matting data, these approaches tend to overfit low-level details in wrong regions, lack generalization to objects with complex structures and real-world scenes such as shadows, as well as suffer from interference of background lines or textures. To address these challenges, in this paper, we propose a novel auxiliary learning framework for mask-guided matting models, incorporating three auxiliary tasks: semantic segmentation, edge detection, and background line detection besides matting, to learn different and effective representations from different types of data and annotations. Our framework and model introduce the following key aspects: (1) to learn real-world adaptive semantic representation for objects with diverse and complex structures under real-world scenes, we introduce extra semantic segmentation and edge detection tasks on more diverse real-world data with segmentation annotations; (2) to avoid overfitting on low-level details, we propose a module to utilize the inconsistency between learned segmentation and matting representations to regularize detail refinement; (3) we propose a novel background line detection task into our auxiliary learning framework, to suppress interference of background lines or textures. In addition, we propose a high-quality matting benchmark, Plant-Mat, to evaluate matting methods on complex structures. Extensively quantitative and qualitative results show that our approach outperforms state-of-the-art mask-guided methods.

Edge-Sensitive Human Cutout with Hierarchical Granularity and Loopy Matting Guidance.

Learning hierarchical poselets for human parsing

Discriminative Hierarchical Part-Based Models for Human Parsing and Action Recognition.

Finer-Net: Cascaded Human Parsing with Hierarchical Granularity

Robust Human Matting via Semantic Guidance

Semantic Human Matting

User-Guided Deep Human Image Matting Using Arbitrary Trimaps

PP-Matting: High-Accuracy Natural Image Matting

Differentiable Multi-Granularity Human Parsing

Attention-Guided Hierarchical Structure Aggregation for Image Matting

Boosting Semantic Human Matting with Coarse Annotations.

SGM-Net: Semantic Guided Matting Net

Human Selective Matting

Attention-guided Temporally Coherent Video Object Matting

Towards Label-Efficient Human Matting: A Simple Baseline for Weakly Semi-Supervised Trimap-Free Human Matting

Human Selective MattingJust Accepted

Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing

Learning Multiple Representations with Inconsistency-Guided Detail Regularization for Mask-Guided Matting

Human Parsing with Edge Enhancement.

Cascade Image Matting with Deformable Graph Refinement

MaGGIe: Masked Guided Gradual Human Instance Matting