Abstract:In modern video coding standards, block-based inter prediction is widely adopted, which brings high compression efficiency. However, in natural videos, there are usually multiple moving objects of arbitrary shapes, resulting in complex motion fields that are difficult to represent compactly. This problem has been tackled by more flexible block partitioning methods in the Versatile Video Coding (VVC) standard, but the more flexible partitions require more overhead bits to signal and still cannot be made arbitrarily shaped. To address this limitation, we propose an object segmentation-assisted inter prediction method (SAIP), where objects in the reference frames are segmented by some advanced technologies. With a proper indication, the object segmentation mask is translated from the reference frame to the current frame as the arbitrary-shaped partition of different regions without any extra signal. Using the segmentation mask, motion compensation is separately performed for different regions, achieving higher prediction accuracy. The segmentation mask is further used to code the motion vectors of different regions more efficiently. Moreover, the segmentation mask is considered in the joint rate-distortion optimization for motion estimation and partition estimation to derive the motion vector of different regions and partition more accurately. The proposed method is implemented into the VVC reference software, VTM version 12.0. Experimental results show that the proposed method achieves up to 1.98%, 1.14%, 0.79%, and on average 0.82%, 0.49%, 0.37% BD-rate reduction for common test sequences, under the Low-delay P, Low-delay B, and Random Access configurations, respectively.

High Efficient Sprite Coding with Directional Spatial Prediction

Sprite Generation For Frame-Based Video Coding

On-Line Sprite Encoding with Large Global Motion Estimation.

Fast Approach of Sprite Coding for Video Content

THE RESEARCH AND IMPROVEMENT OF ONLINE SPRITE ENCODING

Efficient Video Coding with Fractional Resolution Sprite Prediction Technique

A novel coefficient scanning scheme for directional spatial prediction-based image compression

A Novel Method of Sprite Generation Based on Pixel Similarity.

Horizontal Spatial Prediction for High Dimension Intra Coding.

A Pixel-Wise Directional Intra Prediction Method

Video Coding with Spatio-Temporal Texture Synthesis

Improving Lossless Intra Coding of H.264/AVC by Pixel-Wise Spatial Interleave Prediction

Extended application of scalable video coding methods

Probability-Based Fast Intra Prediction Algorithm for Spatial SHVC

Dct-Prediction Based Progressive Fine Granularity Scalable Coding

Improved fine granular scalable coding with interlayer prediction

Object Segmentation-Assisted Inter Prediction for Versatile Video Coding

A Progressive Background Updating Based Coding Scheme for Surveillance Videos.

Hybrid Strategies for Efficient Intra Prediction in Spatial SHVC

In-Scale Motion Compensation for Spatially Scalable Video Coding

Improved intra prediction for high definition video using localized horizontal spatial prediction