Deep Video Stabilization Via Robust Homography Estimation

Weiqing Yan,Yiqiu Sun,Wujie Zhou,Zhaowei Liu,Runmin Cong

DOI: https://doi.org/10.1109/lsp.2023.3329436

2023-01-01

IEEE Signal Processing Letters

Abstract:Video stabilization can improve the visual quality of videos that have been captured on mobile devices or other handheld cameras, which are more prone to shaking and motion artifacts. Most of the existing deep video stabilization methods adopts optical flow-based, which produce artifacts and distortions caused by pixel-level warping and enquire expensive computation time. In this letter, we present a novel unsupervised deep video stabilization approach that addresses the influence of moving objects on video stabilization through robust homography estimation. Specifically, we design a foreground mask estimation module as a preprocessing step using a pre-trained semantic segmentation guided method to distinguish the foreground and background regions, enabling us to estimate camera motion via analyzing the background motion. Additionally, we design a low-level confidence feature extraction module to improve motion alignment loss and ensure robust motion estimation. By integrating the learned low-level confidence features with the foreground mask, we can then design a motion estimation module that captures the consistent spatial correspondence between frames through local and global feature extraction. At last, the learnt robust homography is leveraged to stabilize videos. Our method outperforms related state-of-the-art approaches in both quality and quantity on three public benchmarks while remaining computationally efficient.