A statistical and computational theory for the art of painting
Song-Chun Zhu,Mingtian Zhao
2011-01-01
Abstract:The art of painting has long been a popular form of visual arts among people of different times, places, cultures, professions, etc. Besides the paintings themselves, the art theories and practices behind the pictures are also of interest to not only the artists but many among the rest of us. In particular, the aesthetic mechanism of the art of painting has attracted wide attentions. Recently, researchers started to study the creation and appreciation of paintings using scientific methods. Among these researches, the studies of painterly rendering techniques are very inspiring, which reveal the science behind the art of painting to a good extent by simulating the practices of artists on the computer. But as methods of non-photorealistic computer graphics, most of these techniques focus on specific rendering problems only, while we still look forward to a systematic understanding and a mathematical description of both the creation and appreciation aspects of the art of painting. Towards this objective, we develop a statistical and computational theory for the art of painting based on our studies of painterly rendering and image understanding. Specifically, we model (1) The creation of paintings as stochastic processes of the paintbrush strokes, and (2) The appreciation of paintings as statistical computing processes for image understanding. This dissertation presents our theory and methods by studying the following three concrete problems which constitute a systematic painterly rendering solution. (1) Brush Modeling. We model the basic painterly rendering elements using an example-based method, which has a brush dictionary with probabilistic mapping relations between brush attributes and image semantics. (2) Stroke Placement. To determine where and how the brush strokes should be applied onto the canvas, we propose two stroke placement methods for generic objects and human faces, respectively, the latter being an example of highly structural objects. For generic objects, we apply a parametric method based on spatial point processes and stochastic reaction-diffusion on graphical models. For human faces, we apply a non-parametric template-based method using portrait examples created by artists. (3) Abstract Painting. In addition to the creation of paintings studied in the two problems above, to further understand the appreciation of the art of painting, we study abstract paintings which are characterized by their ambiguities for visual perception. Based on a set of methods for statistical inference of image semantics, we develop a system for rendering abstract paintings with similar effects to those created by artists.