Complex Video Completion Fusing Low-RankBackground and Deep Foreground Priors
Jian-Li Wang,Ting-Zhu Huang,Xi-Le Zhao,Yu-Chun Miao
DOI: https://doi.org/10.1109/LSP.2023.3236585
2022-01-01
Abstract:Recently, low-rank prior (LRP) has achieved success in tensor completion, thanks to its ability in capturing global correlations. However, since the presence of dynamic foreground breaks the low-rank assumption, LRP sometimes fails to work for complex videos. How to effectively capture the implicit low-rankness is a tricky challenge in complex video completion. To address this challenge, we propose a video completion model, which organically combines the respective merits of deterministic LRP and deep image prior. Specifically, we first decompose a complex video into the dynamic foreground and static background, and then characterize each component individually. For the dynamic foreground, instead of the hand-crafted prior, we consider the deep foreground prior expressed by U-Net architecture in an unsupervised manner, while we consider the tensor low-rank prior for the static background. Moreover, we develop an efficient alternating direction method of multipliers algorithm under the plug-and-play framework to tackle the resulting model. Different from the classic plug-and-play framework with a pre-trained and fixed network, our algorithm can adaptively update the parameters of network during iterations, which enables the network to have a more flexible expression ability. Experimental results verify that the proposed method can do better than some state-of-the-art tensor completion methods in complex video completion.