Abstract:The contrast sensitivity function (CSF) is a fundamental signature of the visual system that has been measured extensively in several species. It is defined by the visibility threshold for sinusoidal gratings at all spatial frequencies. Here, we investigated the CSF in deep neural networks using the same 2AFC contrast detection paradigm as in human psychophysics. We examined 240 networks pretrained on several tasks. To obtain their corresponding CSFs, we trained a linear classifier on top of the extracted features from frozen pretrained networks. The linear classifier is exclusively trained on a contrast discrimination task with natural images. It has to find which of the two input images has higher contrast. The network's CSF is measured by detecting which one of two images contains a sinusoidal grating of varying orientation and spatial frequency. Our results demonstrate characteristics of the human CSF are manifested in deep networks both in the luminance channel (a band-limited inverted U-shaped function) and in the chromatic channels (two low-pass functions of similar properties). The exact shape of the networks' CSF appears to be task-dependent. The human CSF is better captured by networks trained on low-level visual tasks such as image-denoising or autoencoding. However, human-like CSF also emerges in mid- and high-level tasks such as edge detection and object recognition. Our analysis shows that human-like CSF appears in all architectures but at different depths of processing, some at early layers, while others in intermediate and final layers. Overall, these results suggest that (i) deep networks model the human CSF faithfully, making them suitable candidates for applications of image quality and compression, (ii) efficient/purposeful processing of the natural world drives the CSF shape, and (iii) visual representation from all levels of visual hierarchy contribute to the tuning curve of the CSF, in turn implying a function which we intuitively think of as modulated by low-level visual features may arise as a consequence of pooling from a larger set of neurons at all levels of the visual system.

Contrast Sensitivity Function Calibration Based on Image Quality Prediction

A Study of Spatial Chromatic Contrast Sensitivity Based on Different Colour Background

Contrast sensitivity function based on color difference under different fields

Are Spatial Chromatic Contrast Sensitivity Band-pass or Lowpass Functions?

Contrast sensitivity function in deep networks

A New Research on Contrast Sensitivity Function Based on Three-Dimensional Space.

A New Research on Contrast Sensitivity Function in 3D Space.

Contrast response function estimation with nonparametric Bayesian active learning

A Study of Spatial Chromatic Contrast Sensitivity Based on Different Colors, Luminance, and Stimulus Patterns

Scale Contrastive Learning with Selective Attentions for Blind Image Quality Assessment

An adaptive calibration algorithm for projected images in daily environment.

Human Visual System-Based Fundus Image Quality Assessment of Portable Fundus Camera Photographs.

Image Quality Assessment and Color Difference

A Method for Color Calibration Based on Simulated Annealing Optimization

A New Image Coding Quality Assessment.

Perceptual Constancy Constrained Single Opinion Score Calibration for Image Quality Assessment

Active Mutual Conjoint Estimation of Multiple Contrast Sensitivity Functions

MEASUREMENT OF FASTING GASTRIC SECRETION.

Measuring the Contrast Sensitivity Function Using the Qcsf Method with 10 Digits

Image Quality Assessment Based on S-Cielab Model

Contrast and visual saliency similarity-induced index for assessing image quality