A Mathematical Theory of Textons and Primal Sketch: Integrating Generative and Descriptive Methods

Cheng-En Guo,Song-Chun Zhu

2005-01-01

Abstract:Textons, coined by Julesz in the 1960s, refer to the atomic structures in natural images and are considered the basic elements in early (pre-attentive) visual perception. The primal sketches, conjectured by Marr in the 1970s, refer to parsimonious token representation of generic images. Both concepts are fundamental to computer and human vision studies. However they have been very elusive due to the lack of rigorous mathematical definitions and models. The objective of this dissertation is to study a mathematical framework for the two concepts and thus to build a theoretical foundation for the early vision theory. Our study leads to generative image models for generic natural images and computing algorithms for learning, simulation, and inference. The framework integrates two main statistical modeling paradigms in the literature. (i) Hierarchic generative methods, such as transformed component analysis, wavelet coding, and sparse coding; and (ii) Descriptive methods, such as Markov random fields, graphical models, and minimax entropy learning. Generally speaking, the former decompose images into image elements, and the latter represent spatial relationships between the components. The dissertation makes the following contributions. (1) We define textons and image primitives as elements in the dictionaries of generative models which are learned from natural images through maximum likelihood estimation. (2) We study a Gestalt field for modeling the spatial relations among textons with reconfigurable neighborhoods. (3) We present a primal sketch model by dividing an image lattice into structural parts as textural parts which are modeled by the generative models and descriptive models respectively. (4) We study the information scaling phenomena and the transitions between various regimes of image representations, for example, the transition between textons and textures.