Abstract:Responses to natural stimuli in area V4—a mid-level area of the visual ventral stream—are well predicted by features from convolutional neural networks (CNNs) trained on image classification. This result has been taken as evidence for the functional role of V4 in object classification. However, we currently do not know if and to what extent V4 plays a role in solving other computational objectives. Here, we investigated normative accounts of V4 (and V1 for comparison) by predicting macaque single-neuron responses to natural images from the representations extracted by 23 CNNs trained on different computer vision tasks including semantic, geometric, 2D, and 3D types of tasks. We found that V4 was best predicted by semantic classification features and exhibited high task selectivity, while the choice of task was less consequential to V1 performance. Consistent with traditional characterizations of V4 function that show its high-dimensional tuning to various 2D and 3D stimulus directions, we found that diverse non-semantic tasks explained aspects of V4 function that are not captured by individual semantic tasks. Nevertheless, jointly considering the features of a pair of semantic classification tasks was sufficient to yield one of our top V4 models, solidifying V4's main functional role in semantic processing and suggesting that V4's selectivity to 2D or 3D stimulus properties found by electrophysiologists can result from semantic functional goals. The functional role of area V4 in the primate visual cortex has been traditionally studied by measuring tuning properties to simple, parametric stimuli, potentially overlooking other important aspects that could be revealed with richer, natural stimuli. Here, we combine single-cell recordings of macaque V1 and V4 responses to natural images, and deep learning models trained on multiple computer vision tasks. We found that V4 responses are best predicted by representations that are critical to solve semantic tasks like object and scene classification. Moreover, our results suggest that that V4's affinity to different 2D and 3D stimulus properties likely stems from its involvement in semantic processing. Overall, our diverse task-driven modeling approach enriches our understanding of the functional role of visual areas in the brain.

V4 Neural Network Model for Shape-Based Feature Extraction and Object Discrimination

V4 neural network model for visual saliency and discriminative local representation of shapes

V4 Shape Features for Contour Representation and Object Detection

A Model of V4 Neurons Based on Sparse Coding.

Shape Representation and Recognition in High Dimensional Feature Space

Hypercolumn-array based image representation and its application to shape-based object detection.

Local Image Descriptor Inspired by Visual Cortex

Learning a model of shape selectivity in V4 cells reveals shape encoding mechanisms in the brain

An Image Local Feature Representation Method Using Hierarchical Vision Network Model

Shape Description and Recognition Method Inspired by the Primary Visual Cortex

A Bio-inspired Model for Image Representation and Image Analysis

Modeling Response Properties of V2 Neurons Using a Hierarchical K-means Model

A SOM Network Model for Feature Extraction by Hyper Columns Architecture of Primary Visual Cortex

Analyzing 3D Model Visual Features Based on Neural Networks

PVRNet: Point-View Relation Neural Network for 3D Shape Recognition

Object Recognition Using a Bio-Inspired Neuron Model with Bottom-Up and Top-Down Pathways

Invariant Visual Object Recognition: Biologically Plausible Approaches

A Hierarchical Model Based on Visual Cortex

A Retina-LGN-V1 Structure-like Spiking Neuron Network for Image Feature Extraction

Predictive Coding in Area V4: Dynamic Shape Discrimination under Partial Occlusion

Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks