High-Resolution Deep Image Matting

Haichao Yu,Ning Xu,Zilong Huang,Yuqian Zhou,Humphrey Shi

DOI: https://doi.org/10.48550/arXiv.2009.06613

2021-01-15

Abstract:Image matting is a key technique for image and video editing and composition. Conventionally, deep learning approaches take the whole input image and an associated trimap to infer the alpha matte using convolutional neural networks. Such approaches set state-of-the-arts in image matting; however, they may fail in real-world matting applications due to hardware limitations, since real-world input images for matting are mostly of very high resolution. In this paper, we propose HDMatt, a first deep learning based image matting approach for high-resolution inputs. More concretely, HDMatt runs matting in a patch-based crop-and-stitch manner for high-resolution inputs with a novel module design to address the contextual dependency and consistency issues between different patches. Compared with vanilla patch-based inference which computes each patch independently, we explicitly model the cross-patch contextual dependency with a newly-proposed Cross-Patch Contextual module (CPC) guided by the given trimap. Extensive experiments demonstrate the effectiveness of the proposed method and its necessity for high-resolution inputs. Our HDMatt approach also sets new state-of-the-art performance on Adobe Image Matting and AlphaMatting benchmarks and produce impressive visual results on more real-world high-resolution images.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the technical challenges of image matting on high - resolution images. Specifically, the existing deep - learning - based image matting methods face hardware limitations when processing high - resolution images, such as insufficient GPU memory, which makes it impossible to directly process these images. In addition, common solution strategies such as down - sampling the input or simple patch - wise inference will lead to problems of detail loss or inconsistency between patches. These problems limit the performance of existing methods in practical applications, especially when dealing with ultra - high - definition images.

To solve the above problems, the author proposes HDMatt, a new deep - learning - based high - resolution image matting method. HDMatt divides the input image into small patches and introduces a novel cross - patch contextual module (CPC) to explicitly model the long - distance contextual dependencies between different patches. This method not only solves the hardware limitation problems encountered by traditional methods when processing high - resolution images, but also improves the quality of the matting results. Especially in the case of large unknown areas, it can more effectively propagate information and improve the accuracy of matting.

In summary, the main contribution of this paper is to propose a new image matting method suitable for high - resolution images. Through the innovative cross - patch context modeling technology, it significantly improves the matting quality and efficiency, making it more practical in practical applications.