Abstract:Dynamic texture refers to the content in video sequences that is characterized by spatial repetition and temporal variation, such as swaying foliage and flowing water. It is a great challenge to compress the dynamic textures efficiently in the current prediction/transform hybrid video coding framework. However, these textures have little information for machine vision, and human visual perception is less sensitive to the textures than to the structures. Thus, we propose a spatiotemporal generative adversarial network (GAN) based dynamic texture synthesis method for surveillance video coding. We detect and remove the dynamic texture content at encoder side, which is irrelevant to machine vision. We generate the dynamic texture content using the proposed GAN at decoder side, so that the reconstructed videos can be observed by human without deteriorating perceptual quality. Specifically, we design a GAN network to synthesize dynamic textures by exploiting the correlation between spatial and temporal neighbors; we present a surveillance video coding scheme with the dynamic texture detection/synthesis method; we build a high-quality dynamic texture dataset, and we collect a dynamic texture testing dataset that goes beyond the existing video coding test datasets by focusing on surveillance scenes. The proposed video coding scheme has been implemented on top of the High Efficiency Video Coding (HEVC) reference software. Experiments have been conducted to evaluate the quantitative and qualitative performance of the proposed coding scheme. Our method achieves 7.4% and 7.6% bit-rate savings in low-delay-B and low-delay-P settings, respectively, at similar visual quality levels in comparison with HEVC.

Video coding by texture analysis and synthesis using graph cut

Video Coding with Spatio-Temporal Texture Synthesis

A Pixel-Level Segmentation-Synthesis Framework for Dynamic Texture Video Compression

Synthesis-Aware Region-Based 3D Video Coding.

Smoothed Reference Inter-Layer Texture Prediction For Bit Depth Scalable Video Coding

Flow-grounded Dynamic Texture Synthesis for Video Compression

Surveillance Video Coding with Dynamic Textural Background Detection

Spatiotemporal Generative Adversarial Network-Based Dynamic Texture Synthesis for Surveillance Video Coding

Generic video coding with abstraction and detail completion

AV1 Video Coding Using Texture Analysis With Convolutional Neural Networks

Applications of just-noticeable depth difference model in joint multiview video plus depth coding

A Novel Scalable Texture Video Coding Scheme With Gpca

Multi-view Video Coding Based on View Prediction

Joint Bit Allocation for 3D Video Coding Based on Virtual View Distortion

Selective VS-MRF-ME and intra coding in H.264 based on spatiotemporal continuity of motion field

Object Segmentation Using Graph Cuts for the H.264 Compressed Video with Moving Background

Joint Texture And Depth Map Video Coding Based On The Scalable Extension Of H.264/Avc

Locally Refined Motion Compensation for Future Video Coding

Lightweight content-adaptive coding in joint analyzing-encoding framework

Fast mode selection based on texture analysis and local motion activity in H.264/JVT

Joint Feature and Texture Coding: Toward Smart Video Representation Via Front-End Intelligence