Abstract:This paper presents a class of statistical models that integrate two statistical modeling paradigms in the literature: (I) Descriptive methods, such as Markov random fields and minimax entropy learning (Zhu, S.C., Wu, Y.N., and Mumford, D. 1997. Neural Computation, 9(8)), and (II) Generative methods, such as principal component analysis, independent component analysis (Bell, A.J. and Sejnowski, T.J. 1997. Vision Research, 37:3327–3338), transformed component analysis (Frey, B. and Jojic, N. 1999. ICCV), wavelet coding (Mallat, S. and Zhang, Z. 1993. IEEE Trans. on Signal Processing, 41:3397–3415; Chen, S., Donoho, D., and Saunders, M.A. 1999. Journal on Scientific Computing, 20(1):33–61), and sparse coding (Olshausen, B.A. and Field, D.J. 1996. Nature, 381:607–609; Lewicki, M.S. and Olshausen, B.A. 1999. JOSA, A. 16(7):1587–1601). In this paper, we demonstrate the integrated framework by constructing a class of hierarchical models for texton patterns (the term “texton” was coined by psychologist Julesz in the early 80s). At the bottom level of the model, we assume that an observed texture image is generated by multiple hidden “texton maps”, and textons on each map are translated, scaled, stretched, and oriented versions of a window function, like mini-templates or wavelet bases. The texton maps generate the observed image by occlusion or linear superposition. This bottom level of the model is generative in nature. At the top level of the model, the spatial arrangements of the textons in the texton maps are characterized by minimax entropy principle, which leads to embellished versions of Gibbs point process models (Stoyan, D., Kendall, W.S., and Mecke, J. 1985. Stochastic Geometry and its Applications). The top level of the model is descriptive in nature. We demonstrate the integrated model by a set of experiments.

A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models

Incorporating Probabilistic Knowledge into Topic Models.

Modeling Visual Patterns by Integrating Descriptive and Generative Methods.

Probabilistic Learning of Partial Least Squares Regression Model: Theory and Industrial Applications

Association Pattern Discovery Via Theme Dictionary Models

A Probabilistic Model for Image Representation Via Multiple Patterns

Versatile Energy-Based Probabilistic Models for High Energy Physics

Generative vs. Discriminative modeling under the lens of uncertainty quantification

Probabilistic Tangent Subspace

A Probabilistic Generative Model of Linguistic Typology

Hidden Variable Models in Text Classification and Sentiment Analysis

Building Probabilistic Models for Natural Language

A neural probabilistic language model

A Probabilistic Model with Commonsense Constraints for Pattern-based Temporal Fact Extraction

Identification of Probabilities

Graphical Models, Exponential Families, and Variational Inference

Automatic Generation of Probabilistic Programming from Time Series Data

A Statistical Turing Test for Generative Models

Predictive Multiplicity in Probabilistic Classification

Sparse and Deep Generalizations of the FRAME Model

Selection of sequence motifs and generative Hopfield-Potts models for protein familiesilies