Deep Image Matting: A Comprehensive Survey

Jizhizi Li,Jing Zhang,Dacheng Tao

2023-04-10

Abstract:Image matting refers to extracting precise alpha matte from natural images, and it plays a critical role in various downstream applications, such as image editing. Despite being an ill-posed problem, traditional methods have been trying to solve it for decades. The emergence of deep learning has revolutionized the field of image matting and given birth to multiple new techniques, including automatic, interactive, and referring image matting. This paper presents a comprehensive review of recent advancements in image matting in the era of deep learning. We focus on two fundamental sub-tasks: auxiliary input-based image matting, which involves user-defined input to predict the alpha matte, and automatic image matting, which generates results without any manual intervention. We systematically review the existing methods for these two tasks according to their task settings and network structures and provide a summary of their advantages and disadvantages. Furthermore, we introduce the commonly used image matting datasets and evaluate the performance of representative matting methods both quantitatively and qualitatively. Finally, we discuss relevant applications of image matting and highlight existing challenges and potential opportunities for future research. We also maintain a public repository to track the rapid development of deep image matting at <a class="link-external link-https" href="https://github.com/JizhiziLi/matting-survey" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to provide a comprehensive review of recent deep learning-based image matting methods and to discuss them in detail compared to traditional methods. Specifically:

1. **Problem Definition**:
- Image matting refers to the precise extraction of the foreground object's transparency mask (alpha matte) from a natural image. This technology plays a crucial role in applications such as image editing, e-commerce platform advertising, online video conference background replacement, and virtual reality.

2. **Research Background**:
- Although image matting is an ill-posed problem, traditional methods have been attempting to solve this challenge for decades. Early methods relied on various auxiliary user inputs (such as trimap or scribbles) to alleviate the challenge. With the development of deep learning, new techniques have continuously emerged, including automatic matting, interactive matting, and reference image matting.

3. **Main Contributions**:
- The paper provides a comprehensive review of deep learning-based image matting methods, focusing on two fundamental sub-tasks: auxiliary input-based image matting and automatic image matting.
- It systematically reviews existing methods and classifies them based on task settings and network structures, summarizing their advantages and disadvantages.
- It introduces commonly used image matting datasets and conducts quantitative and qualitative evaluations of representative methods.
- It explores related applications of image matting and points out current challenges and future research directions.

4. **Method Classification**:
- **Auxiliary Input-based Image Matting**: Predicts the alpha matte through user-defined inputs, such as trimap, scribbles, background images, etc.
- **Automatic Image Matting**: Generates results without any manual intervention, usually predicting for specific foreground objects.

Through these reviews and analyses, the paper provides researchers and practitioners with a comprehensive understanding of the latest technologies and potential research opportunities in the current field of image matting.