A Multi-Level Affinity-Based Matting and Compositing Method Based on a Joint Solution of Foreground and Transparency
Guilin YAO,Biying ZHANG,Yanrong ZHANG,Xiaodong SU
DOI: https://doi.org/10.1360/ssi-2020-0121
2021-01-01
Scientia Sinica Informationis
Abstract:Digital image matting and compositing is a classical problem in virtual reality. According to the matting formula, the matting problem needs to accurately extract the alpha value and the color value of the foreground object from the input image, while the compositing problem only uses the above matting results to combine with the new background image to make it a new synthetic image. However, at present, plenty of matting algorithms are only limited to the calculation of alpha value in matting, and ignore the calculation of foreground color used in the subsequent compositing steps. In fact, the above way is only limited to the theoretical research, and is separated from the compositing application level in the matting problem. Meanwhile, the effect of other algorithms for simultaneous foreground color calculation is still unsatisfying, and there is still a lack of compositing image data sets and comparison of compositing images. In this paper, we make full use of the advantages of the flexible search range in the affinity-based method, and use the three-level structure of the opaque pixels and the mixed pixels for hierarchical processing to jointly solve the final alpha value and foreground color. Among them, each level uses the forward level priors along with the pixel correlation of the affinity method and color fitting constraints, and finally solves the problem in a closed form solution. In the experiments, this paper not only compares the results in alpha level, but also introduces the comparison of compositing image level in the field of digital matting for the first time. In the compositing layer, white backgrounds and corresponding natural backgrounds are both introduced for the input image and trimap of each size. At the same time, in each method, the real alpha value and foreground color are generally composited with these backgrounds, and a simple alpha level composition is also carried out. The experimental results show that, in terms of the quantitative evaluation and visual quality, the final alpha results and the above compositing effects of this method are better than the state-of-the-art matting and compositing methods as well as some newly generated methods.