Abstract:Responses to natural stimuli in area V4—a mid-level area of the visual ventral stream—are well predicted by features from convolutional neural networks (CNNs) trained on image classification. This result has been taken as evidence for the functional role of V4 in object classification. However, we currently do not know if and to what extent V4 plays a role in solving other computational objectives. Here, we investigated normative accounts of V4 (and V1 for comparison) by predicting macaque single-neuron responses to natural images from the representations extracted by 23 CNNs trained on different computer vision tasks including semantic, geometric, 2D, and 3D types of tasks. We found that V4 was best predicted by semantic classification features and exhibited high task selectivity, while the choice of task was less consequential to V1 performance. Consistent with traditional characterizations of V4 function that show its high-dimensional tuning to various 2D and 3D stimulus directions, we found that diverse non-semantic tasks explained aspects of V4 function that are not captured by individual semantic tasks. Nevertheless, jointly considering the features of a pair of semantic classification tasks was sufficient to yield one of our top V4 models, solidifying V4's main functional role in semantic processing and suggesting that V4's selectivity to 2D or 3D stimulus properties found by electrophysiologists can result from semantic functional goals. The functional role of area V4 in the primate visual cortex has been traditionally studied by measuring tuning properties to simple, parametric stimuli, potentially overlooking other important aspects that could be revealed with richer, natural stimuli. Here, we combine single-cell recordings of macaque V1 and V4 responses to natural images, and deep learning models trained on multiple computer vision tasks. We found that V4 responses are best predicted by representations that are critical to solve semantic tasks like object and scene classification. Moreover, our results suggest that that V4's affinity to different 2D and 3D stimulus properties likely stems from its involvement in semantic processing. Overall, our diverse task-driven modeling approach enriches our understanding of the functional role of visual areas in the brain.

V4 neural network model for visual saliency and discriminative local representation of shapes

V4 Neural Network Model for Shape-Based Feature Extraction and Object Discrimination

A Model of V4 Neurons Based on Sparse Coding.

V4 Shape Features for Contour Representation and Object Detection

Saliency Prediction on Omnidirectional Images with Brain-Like Shallow Neural Network.

Learning a model of shape selectivity in V4 cells reveals shape encoding mechanisms in the brain

Multiscale Brain-Like Neural Network for Saliency Prediction on Omnidirectional Images

Visually-salient contour detection using a V1 neural model with horizontal connections

A neural computational model for bottom-up attention with invariant and overcomplete representation

Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks

Toward a Unified Theory of Visual Area V4

Shape Selectivity of Middle Superior Temporal Sulcus Body Patch Neurons

Computational Constraints Underlying the Emergence of Functional Domains in the Topological Map of Macaque V4

Local Image Descriptor Inspired by Visual Cortex

Representation of Natural Contours by a Neural Population in Monkey V4

Deep supervised visual saliency model addressing low-level features

Improve Neural Network Using Saliency.

An Image Local Feature Representation Method Using Hierarchical Vision Network Model

Predictive Coding in Area V4: Dynamic Shape Discrimination under Partial Occlusion

Clustered Functional Domains for Curves and Corners in Cortical Area V4

Shape Description and Recognition Method Inspired by the Primary Visual Cortex