IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation

Lingtong Kong,Boyuan Jiang,Donghao Luo,Wenqing Chu,Xiaoming Huang,Ying Tai,Chengjie Wang,Jie Yang

DOI: https://doi.org/10.1109/cvpr52688.2022.00201

2022-01-01

Abstract:Prevailing video frame interpolation algorithms, that generate the intermediate frames from consecutive inputs, typically rely on complex model architectures with heavy parameters or large delay, hindering them from diverse real-time applications. In this work, we devise an efficient encoder-decoder based network, termed IFRNet, for fast in-termediate frame synthesizing. It first extracts pyramid features from given inputs, and then refines the bilateral in-termediate flow fields together with a powerful intermedi-ate feature until generating the desired output. The gradu-ally refined intermediate feature can not only facilitate in-termediate flow estimation, but also compensate for con-textual details, making IFRNet do not need additional syn-thesis or refinement module. To fully release its potential, we further propose a novel task-oriented optical flow dis-tillation loss to focus on learning the useful teacher knowl-edge towards frame synthesizing. Meanwhile, a new ge-ometry consistency regularization term is imposed on the gradually refined intermediate features to keep better structure layout. Experiments on various benchmarks demon-strate the excellent performance and fast inference speed of proposed approaches. Code is available at https://github.com/ltkong218/IFRNet.