Abstract:Image stylization has seen significant advancement and widespread interest over the years, leading to the development of a multitude of techniques. Extending these stylization techniques, such as Neural Style Transfer (NST), to videos is often achieved by applying them on a per-frame basis. However, per-frame stylization usually lacks temporal consistency, expressed by undesirable flickering artifacts. Most of the existing approaches for enforcing temporal consistency suffer from one or more of the following drawbacks: They (1) are only suitable for a limited range of techniques, (2) do not support online processing as they require the complete video as input, (3) cannot provide consistency for the task of stylization, or (4) do not provide interactive consistency control. Domain-agnostic techniques for temporal consistency aim to eradicate flickering completely but typically disregard aesthetic aspects. For stylization tasks, however, consistency control is an essential requirement as a certain amount of flickering adds to the artistic look and feel. Moreover, making this control interactive is paramount from a usability perspective. To achieve the above requirements, we propose an approach that stylizes video streams in real-time at full HD resolutions while providing interactive consistency control. We develop a lite optical-flow network that operates at 80 FPS on desktop systems with sufficient accuracy. Further, we employ an adaptive combination of local and global consistency features and enable interactive selection between them. Objective and subjective evaluations demonstrate that our method is superior to state-of-the-art video consistency approaches.

Real-time Directional Stylization of Images and Videos.

Video Stylization: Painterly Rendering and Optimization With Content Extraction

Diverse Image Style Transfer Via Invertible Cross-Space Mapping

An adaptive cartoon-like stylization for color video in real time

Online Video Stream Abstraction and Stylization

Real-Time Painterly Abstraction by Pseudo Morphological Filtering

Cartoon-like Stylization of Video for Real-Time Applications.

Interactive Control over Temporal Consistency while Stylizing Video Streams

Automatic Motion-Guided Video Stylization and Personalization

Real-time Feature-Aware Video Abstraction.

Towards 4D Human Video Stylization

Highly Detailed and Temporal Consistent Video Stylization via Synchronized Multi-Frame Diffusion

Real-Time Portrait Stylization on the Edge

Real-time Localized Photorealistic Video Style Transfer

Painterly Animation Using Video Semantics and Feature Correspondence.

Selective Image Abstraction

Real-time Rendering Algorithm Based on a Hybrid Rendering Scheme

Temporal Consistent Oil Painting Video Stylization.

Realtime Fewshot Portrait Stylization Based On Geometric Alignment

Real-time Photo Style Transfer

Frame Difference-Based Real-Time Video Stylization in Video Calls