Abstract:This paper proposes an image-to-painting translation method that generates vivid and realistic painting artworks with controllable styles. Different from previous image-to-image translation methods that formulate the translation as pixel-wise prediction, we deal with such an artistic creation process in a vectorized environment and produce a sequence of physically meaningful stroke parameters that can be further used for rendering. Since a typical vector render is not differentiable, we design a novel neural renderer which imitates the behavior of the vector renderer and then frame the stroke prediction as a parameter searching process that maximizes the similarity between the input and the rendering output. We explored the zero-gradient problem on parameter searching and propose to solve this problem from an optimal transportation perspective. We also show that previous neural renderers have a parameter coupling problem and we re-design the rendering network with a rasterization network and a shading network that better handles the disentanglement of shape and color. Experiments show that the paintings generated by our method have a high degree of fidelity in both global appearance and local textures. Our method can be also jointly optimized with neural style transfer that further transfers visual style from other images. Our code and animated results are available at \url{<a class="link-external link-https" href="https://jiupinjia.github.io/neuralpainter/" rel="external noopener nofollow">this https URL</a>}.

Painterly rendering with vector field based feature extraction

General Virtual Sketching Framework for Vector Line Art

PencilArt: A Chromatic Penciling Style Generation Framework.

Computer-Aided Rendering of Chinese Meticulous Figure Paintings

A New Approch For Automatic Rendering Of Pen-And-Ink Illustrations

Sketch-R2CNN: an Attentive Network for Vector Sketch Recognition

Automatic Image-Based Pencil Sketch Rendering

Automatic Generation of Pen-and-ink Drawings from Photos.

From image parsing to painterly rendering

Example Based Painting Generation

Artistic Portrait Drawing with Vector Strokes

Expressive Rendering Based on Interpretation of Spatial Features

Vectorized Painting with Temporal Diffusion Curves

Vectorization of Line Drawing Image Based on Junction Analysis

Importance Driven Texture-Based Vector Field Visualization

Visual Importance based Painterly Rendering for Images

Real-Time Painterly Abstraction by Pseudo Morphological Filtering

Droplet: A Virtual Brush Model to Simulate Chinese Calligraphy and Painting

PTGCF: Printing Texture Guided Color Fusion for Impressionism Oil Painting Style Rendering

Parallel Pencil Drawing Stylization Via Structure-Aware Optimization

Stylized Neural Painting