Abstract:Dynamic texture refers to the content in video sequences that is characterized by spatial repetition and temporal variation, such as swaying foliage and flowing water. It is a great challenge to compress the dynamic textures efficiently in the current prediction/transform hybrid video coding framework. However, these textures have little information for machine vision, and human visual perception is less sensitive to the textures than to the structures. Thus, we propose a spatiotemporal generative adversarial network (GAN) based dynamic texture synthesis method for surveillance video coding. We detect and remove the dynamic texture content at encoder side, which is irrelevant to machine vision. We generate the dynamic texture content using the proposed GAN at decoder side, so that the reconstructed videos can be observed by human without deteriorating perceptual quality. Specifically, we design a GAN network to synthesize dynamic textures by exploiting the correlation between spatial and temporal neighbors; we present a surveillance video coding scheme with the dynamic texture detection/synthesis method; we build a high-quality dynamic texture dataset, and we collect a dynamic texture testing dataset that goes beyond the existing video coding test datasets by focusing on surveillance scenes. The proposed video coding scheme has been implemented on top of the High Efficiency Video Coding (HEVC) reference software. Experiments have been conducted to evaluate the quantitative and qualitative performance of the proposed coding scheme. Our method achieves 7.4% and 7.6% bit-rate savings in low-delay-B and low-delay-P settings, respectively, at similar visual quality levels in comparison with HEVC.

A Survey of Dynamic Texture Synthesis Technic Research

The Technology of Sampled-based Texture Synthesis

A Novel Method of Dynamic Textures Analysis and Synthesis.

Dynamic texture synthesis using a spatial temporal descriptor

Skeletal Texture Synthesis

Motion Field Texture Synthesis.

Modeling and rendering of realistic waterfall scenes with dynamic texture sprites: Research Articles

Modeling And Rendering Of Realistic Waterfall Scenes With Dynamic Texture Sprites

Dynamic textures using wavelet analysis

Video-based Dynamic Scenes Generation with Compound Micro-Frame

Block Motion Based Dynamic Texture Analysis: A Review

Dynamic Texture Recognition by Spatio-Temporal Multiresolution Histograms

A Pixel-Level Segmentation-Synthesis Framework for Dynamic Texture Video Compression

Depth-image-based Rendering with Spatial and Temporal Texture Synthesis for 3DTV

Kernel Learning for Dynamic Texture Synthesis.

Image and Video Retexturing

A Generative Method for Textured Motion: Analysis and Synthesis

Real-time texture synthesis by patch-based sampling

Asymptotically Admissible Texture Synthesis

Spatiotemporal Generative Adversarial Network-Based Dynamic Texture Synthesis for Surveillance Video Coding