BWIN: A Bilateral Warping Method for Video Frame Interpolation

Fanyong Xue,Jie Li,Jiannan Liu,Chentao Wu

DOI: https://doi.org/10.1109/icme51207.2021.9428254

2021-01-01

Abstract:Flow-based video frame interpolation approaches typically adopt forward or backward warping to approximate the intermediate frames. And a synthesis network is used to refine the interpolation results. Optical flows indicate motion between two input frames, but both forward and backward warping only utilize the first frame. In this work, we propose bilateral warping to make full use of optical flows. Specifically, the proposed bilateral warping yields intermediate candidates from not only the first frame but also the second frame. Our model first applies bilateral warping on the input frames and contextual features. Then, we add skip connections from the input frames and contextual features to the synthesis network. Finally, the synthesis network generates the interpolation results by integrating the original and warped representations. The experimental results on a wide variety of datasets demonstrate the superiority of the proposed approach over the state-of-the-art video frame interpolation methods.