Abstract:Abstract The human visual ability to recognize objects and scenes is widely thought to rely on representations in category-selective regions of visual cortex. These representations could support object vision by specifically representing objects, or, more simply, by representing complex visual features regardless of the particular spatial arrangement needed to constitute real world objects. That is, by representing visual textures. To discriminate between these hypotheses, we leveraged an image synthesis approach that, unlike previous methods, provides independent control over the complexity and spatial arrangement of visual features. We found that human observers could easily detect a natural object among synthetic images with similar complex features that were spatially scrambled. However, observer models built from BOLD responses from category-selective regions, as well as a model of macaque inferotemporal cortex and Imagenet-trained deep convolutional neural networks, were all unable to identify the real object. This inability was not due to a lack of signal-to-noise, as all of these observer models could predict human performance in image categorization tasks. How then might these texture-like representations in category-selective regions support object perception? An image-specific readout from category-selective cortex yielded a representation that was more selective for natural feature arrangement, showing that the information necessary for object discrimination is available. Thus, our results suggest that the role of human category-selective visual cortex is not to explicitly encode objects but rather to provide a basis set of texture-like features that can be infinitely reconfigured to flexibly learn and identify new object categories. Significance Statement Virtually indistinguishable metamers of visual textures, such as wood grain, can be synthesized by matching complex features regardless of their spatial arrangement (1–3). However, humans are not fooled by such synthetic images of scrambled objects. Thus, category-selective regions of human visual cortex might be expected to exhibit representational geometry preferentially sensitive to natural objects. Contrarily, we demonstrate that observer models based on category-selective regions, models of macaque inferotemporal cortex and Imagenet-trained deep convolutional neural networks do not preferentially represent natural images, even while they are able to discriminate image categories. This suggests the need to reconceptualize the role of category-selective cortex as representing a basis set of complex texture-like features, useful for a myriad of visual behaviors.

Perceptual Texture Dimensions Modulate Neuronal Response Dynamics in Visual Cortical Area V4

Neuronal and behavioral responses to naturalistic texture im-ages in macaque monkeys

Neuronal and behavioral responses to naturalistic texture images in macaque monkeys

The representational geometry for naturalistic textures in macaque V1 and V2

A Texture Statistics Encoding Model Reveals Hierarchical Feature Selectivity across Human Visual Cortex

Assessing neuronal correlates of salience and their adaptability with naturalistic textures in macaque V1 and V2

A two-photon imaging study on the roles of macaque V1 and V4 in texture segregation

Responses of neurons in macaque V4 to object and texture images

Developmentally stable representations of naturalistic image structure in macaque visual cortex

Mesoscale functional organization and connectivity of color, disparity, and naturalistic texture in human second visual area

Neural basis of perceptual surface qualities: Evidence from EEG decoding

Neural networks active during tactile form perception: common and differential activity during macrospatial and microspatial tasks

Adaptive Processing and Perceptual Learning in Visual Cortical Areas V1 and V4.

Texture-like representation of objects in human visual cortex

Foundations of visual form selectivity for neurons in macaque V1 and V2

Spatiotemporal Dynamics of Neuronal Population Response in the Primary Visual Cortex.

Mapping the dynamics of visual feature coding: Insights into perception and integration

The Representation Of Shape And Texture In Category‐Selective Regions Of Ventral‐Temporal Cortex

Perceptual Hue, Lightness, and Chroma Are Represented in a Multidimensional Functional Anatomical Map in Macaque V1.

Neural correlates of stimulus spatial frequency-dependent contrast detection