Modulated Variable-Rate Deep Video Compression

Jianping Lin,Dong Liu,Jie Liang,Houqiang Li,Feng Wu

DOI: https://doi.org/10.1109/dcc50243.2021.00058

2021-01-01

Abstract:Rate adaption is one of the decisive factors for the applications of video compression. However, previous deep video compression methods are usually optimized for a single fixed rate-distortion (R-D) tradeoff. While they can achieve multiple bitrates by training multiple independent models, the realized bitrates are limited to several discrete points on the R-D curve and the storage cost increases proportionally to the number of models. In this paper, we propose a variable-rate scheme for deep video compression, which can achieve continuously variable rate by a single model, i.e., it can reach any point on the R-D curve. In our scheme, two deep auto-encoders are used to compress the residual and the motion vector field respectively, which directly generate the final bitstream. The basic rate adaptation can be achieved by using the R-D tradeoff parameter to deeply modulate all the internal feature maps of the auto-encoders. However, other modules in our scheme, notably motion estimation and motion compensation, also affect the final bitrate indirectly. We further use the R-D tradeoff parameter to modulate them via a conditional map, which effectively improves the compression efficiency. We use a multi-rate-distortion loss function together with a step-by-step training strategy to optimize the entire scheme. Our experiments show that the proposed scheme achieves continuously variable rate by a single model with almost the same compression efficiency as multiple fixed-rate models. The additional parameters and computation of our model are negligible when compared with a single fixed-rate model.