Abstract:Computer vision can be considered a highly specialized data collection and data analysis problem. We need to understand the special properties of image data in order to construct statistical models for representing the wide variety of image patterns. One special property of vision that distinguishes itself from other sensory data such as speech data is that distance or scale plays a profound role in image data. More specifically, visual objects and patterns can appear at a wide range of distances or scales, and the same visual pattern appearing at different distances or scales produces different image data with different statistical properties, thus entails different regimes of statistical models. In particular, we show that the entropy rate of the image data changes over the viewing distance (as well as the camera resolution). Moreover, the inferential uncertainty changes with viewing distance too. We call these changes information scaling. From this perspective, we examine both empirically and theoretically two prominent and yet largely isolated research themes in image modeling literature, namely, wavelet sparse coding and Markov random fields. Our results indicate that the two models are appropriate on two different entropy regimes: sparse coding targets the low entropy regime, whereas the random fields are suitable for the high entropy regime. Because of information scaling, both models are necessary for representing and interpreting image intensity patterns in the whole entropy range, and information scaling triggers transitions between these two regimes of models. This motivates us to propose a full-zoom primal sketch model that integrates both sparse coding and Markov random fields. In this model, local image intensity patterns are classified into “sketchable regime” and “non-sketchable regime” by a sketchability criterion. In the sketchable regime, the image data are represented deterministically by highly parametrized sketch primitives. In the non-sketchable regime, the image data are characterized by Markov random fields whose sufficient statistics summarize computational results from failed attempts of sparse coding. The contribution of our work is two folded. First, information scaling provides a dimension to chart the space of natural images. Second, the full-zoom modeling scheme provides a natural integration of sparse coding and Markov random fields, thus enables us to develop a new and richer class of statistical models.

Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields

Unified theory for joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields

Relationships between the degrees of freedom in the affine Gaussian derivative model for visual receptive fields and 2-D affine image transformations, with application to covariance properties of simple cells in the primary visual cortex

Orientation selectivity properties for the affine Gaussian derivative and the affine Gabor models for visual receptive fields

Derivatives and Inverse of Cascaded Linear+Nonlinear Neural Models

Do the receptive fields in the primary visual cortex span a variability over the degree of elongation of the receptive fields?

Covariant spatio-temporal receptive fields for neuromorphic computing

Idealized computational models for auditory receptive fields

The Gabor-Einstein Wavelet: A Model for the Receptive Fields of V1 to MT Neurons

Perception Of Transformation Invariance In The Visual Pathway

Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations

Spatio-Chromatic Information available from different Neural Layers via Gaussianization

The Lie Derivative for Measuring Learned Equivariance

Derivatives and inverse of a linear-nonlinear multi-layer spatial vision model

Development of Invariant Feature Maps Via a Computational Model of Simple and Complex Cells.

A Spatial Concordance Correlation Coefficient with an Application to Image Analysis

Invariant global motion recognition in the dorsal visual system: a unifying theory

Learning Variance Statistics of Natural Images

From Information Scaling of Natural Images to Regimes of Statistical Models

Bipartite invariance in mouse primary visual cortex

Title From Information Scaling of Natural Images to Regimes of Statistical Models Permalink