Abstract:The influence of natural image transformations on receptive field responses is crucial for modelling visual operations in computer vision and biological vision. In this regard, covariance properties with respect to geometric image transformations in the earliest layers of the visual hierarchy are essential for expressing robust image operations, and for formulating invariant visual operations at higher levels. This paper defines and proves a set of joint covariance properties for spatio-temporal receptive fields in terms of spatio-temporal derivative operators applied to spatio-temporally smoothed image data under compositions of spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations. Specifically, the derived relations show how the parameters of the receptive fields need to be transformed, in order to match the output from spatio-temporal receptive fields under composed spatio-temporal image transformations. For this purpose, we also fundamentally extend the notion of scale-normalized derivatives to affine-normalized derivatives, that are computed based on spatial smoothing with affine Gaussian kernels, and analyze the covariance properties of the resulting affine-normalized derivatives for the affine group as well as for important subgroups thereof. We conclude with a geometric analysis, showing how the derived joint covariance properties make it possible to relate or match spatio-temporal receptive field responses, when observing, possibly moving, local surface patches from different views, under locally linearized perspective or projective transformations, as well as when observing different instances of spatio-temporal events, that may occur either faster or slower between different views of similar spatio-temporal events.

Do the receptive fields in the primary visual cortex span a variability over the degree of elongation of the receptive fields?

Orientation selectivity properties for the affine Gaussian derivative and the affine Gabor models for visual receptive fields

Relationships between the degrees of freedom in the affine Gaussian derivative model for visual receptive fields and 2-D affine image transformations, with application to covariance properties of simple cells in the primary visual cortex

Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields

Connectivity and Functional Organization in the Mammalian Visual Cortex

Improve EN model of simple cell combined with non-classical receptive field

Orientation Selectivity in Cat Primary Visual Cortex: Local and Global Measurement

The Dynamics of Visual Responses in the Primary Visual Cortex.

Unified theory for joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields

Extraclassical receptive field phenomena & short-range connectivity in V1

Afferent specificity, feature specific connectivity influence orientation selectivity: A computational study in mouse primary visual cortex

Theory of orientation tuning in visual cortex.

V1 receptive field structure contributes to neuronal response latency

Foundations of visual form selectivity for neurons in macaque V1 and V2

Binocular receptive-field construction in the primary visual cortex

Haphazard wiring of simple receptive fields and orientation columns in visual cortex

Emergence of Orientation Selectivity in the Mammalian Visual Pathway

A sub-Riemannian model of the visual cortex with frequency and phase

Correlation Between Spatial Frequency and Orientation Selectivity in V1 Cortex: Implications of a Network Model.

Revisiting the high-dimensional geometry of population responses in visual cortex