Abstract:In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences [Carandini&Heeger12]. However, the conventional literature is usually too focused on only describing the input->output transform. Instead, here we present the maths of such cascades beyond the forward transform, namely the Jacobians and the inverse. The fundamental reason for this analytical treatment is that it offers useful insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (discrimination regions) and the adaptation of the receptive fields can be seen in the expression of the Jacobian wrt the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian wrt the parameters shows which aspects of the model have bigger impact in the response, and hence bigger relevance. The analytic inverse implies conditions for the response and the model to ensure decoding. From an applied perspective, (a) the Jacobian wrt the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometry, (b) the Jacobian matrices wrt the parameters are convenient to learn the model from classical experiments or alternative optimization goals, and (c) the inverse is a model-based alternative to blind machine-learning neural decoding that does not include meaningful biological information. The theory is checked by building a derivable and invertible vision model that actually follows the modular program suggested by Carandini&Heeger. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization layers are substituted by equivalent layers such as the Wilson-Cowan model at V1, or a tone-mapping model at the retina.

Derivatives and inverse of a linear-nonlinear multi-layer spatial vision model

Derivatives and Inverse of Cascaded Linear+Nonlinear Neural Models

Understanding physiological and degenerative natural vision mechanisms to define contrast and contour operators

A nonlinear elasticity model in computer vision

Nonlinearities and Adaptation of Color Vision from Sequential Principal Curves Analysis

Unified theory for joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields

Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields

Understanding Neural Network Systems for Image Analysis using Vector Spaces and Inverse Maps

Learning Variance Statistics of Natural Images

Mechanisms of Adaptive Spatial Integration in a Neural Model of Cortical Motion Processing

Relationships between the degrees of freedom in the affine Gaussian derivative model for visual receptive fields and 2-D affine image transformations, with application to covariance properties of simple cells in the primary visual cortex

Perception Of Transformation Invariance In The Visual Pathway

Learning Multi-Layer Transform Models

A Multiscale Theory for Image Registration and Nonlinear Inverse Problems

Nonlinear ICA Using Volume-Preserving Transformations.

Two-color optically-addressed spatial light modulator as generic spatio-temporal systems

Cortical-inspired Wilson-Cowan-type equations for orientation-dependent contrast perception modelling

Towards Modeling the Interaction of Spatial-Associative Neural Network Representations for Multisensory Perception

Invariant-based Mapping of Space During General Motion of an Observer

Cortical origins of MacKay-type visual illusions: A case for the non-linearity

Differentiable Visual Computing for Inverse Problems and Machine Learning