Neural organization of higher visual functions

Edmund T. Rolls

DOI: https://doi.org/10.3389/fsurg.2022.950793

IF: 7.07

1991-01-01

Current Opinion in Neurobiology

Abstract:This paper reviews recent progress made with both anatomical and neurophysiological techniques in understanding the neural organization of higher visual functions in V4, the temporal cortical visual areas, and brain regions connected to these, such as the striatum and amygdala.

What problem does this paper attempt to address?

Vision, Emotion and Memory: from Neurophysiology to Computation

ET Rolls

DOI: https://doi.org/10.1016/s0531-5131(03)00981-6

2003-01-01

International Congress Series

Abstract:The inferior temporal visual cortex provides invariant representations of objects. The computations underlying this can be understood in the framework of a hierarchical series of competitive networks in the ventral visual stream which learn invariant representations by using a short-term memory trace to extract properties of the visual input that are invariant over short periods and are thus statistically likely to be from the same object. One set of outputs from the inferior temporal cortex is to the orbitofrontal cortex and amygdala, which are involved in representing primary rewards and punishers (such as taste, pleasant touch, and pain), and in associating previously neutral visual stimuli with these primary reinforcers. In that emotions can be defined as states elicited by reinforcers, these functions of the orbitofrontal cortex and amygdala provide a basis for understanding how emotion is implemented in the brain. A second output of the inferior temporal cortex is to the hippocampal system, which is implicated in episodic memory and in spatial function. A third output of the inferior temporal visual cortex is to the prefrontal cortex, which is involved in short-term memory and attentional processes. The combined neurophysiological and computational approach taken to these issues provides a way of understanding how these systems actually operate in the brain to implement object perception, emotion, and long- and short-term memory.
A mouse model of higher visual cortical function

Lindsey L Glickfeld,R Clay Reid,Mark L Andermann

DOI: https://doi.org/10.1016/j.conb.2013.08.009

IF: 7.07

2014-02-01

Current Opinion in Neurobiology

Abstract:During sensory experience, the retina transmits a diverse array of signals to the brain, which must be parsed to generate meaningful percepts that can guide decisions and actions. Decades of anatomical and physiological studies in primates and carnivores have revealed a complex parallel and hierarchical organization by which distinct visual features are distributed to, and processed by, different brain regions. However, these studies have been limited in their ability to dissect the circuit mechanisms involved in the transformation of sensory inputs into complex cortical representations and action patterns. Multiple groups have therefore pushed to explore the organization and function of higher visual areas in the mouse. Here we review the anatomical and physiological findings of these recent explorations in mouse visual cortex. These studies find that sensory input is processed in a diverse set of higher areas that are each interconnected with specific limbic and motor systems. This hierarchical and parallel organization is consistent with the multiple streams that have been found in the higher visual areas of primates. We therefore propose that the mouse visual system is a useful model to explore the circuits underlying the transformation of sensory inputs into goal-directed perceptions and actions.

neurosciences
A unifying framework for functional organization in early and higher ventral visual cortex

Eshed Margalit,Hyodong Lee,Dawn Finzi,James J DiCarlo,Kalanit Grill-Spector,Daniel L K Yamins,James J. DiCarlo,Daniel L.K. Yamins

DOI: https://doi.org/10.1016/j.neuron.2024.04.018

IF: 16.2

2024-07-18

Neuron

Abstract:Margalit et al. develop a topographic artificial neural network that predicts both functional responses and spatial organization of multiple cortical areas of the primate visual system. In turn, the model minimizes between-area wiring length and produces more brain-like responses to visual stimuli than spatially unconstrained alternative models.

neurosciences
Toward a Unified Theory of Visual Area V4

Anna W. Roe,Leonardo Chelazzi,Charles E. Connor,Bevil R. Conway,Ichiro Fujita,Jack L. Gallant,Haidong Lu,Wim Vanduffel

DOI: https://doi.org/10.1016/j.neuron.2012.03.011

IF: 16.2

2012-01-01

Neuron

Abstract:Visual area V4 is a midtier cortical area in the ventral visual pathway. It is crucial for visual object recognition and has been a focus of many studies on visual attention. However, there is no unifying view of V4's role in visual processing. Neither is there an understanding of how its role in feature processing interfaces with its role in visual attention. This review captures our current knowledge of V4, largely derived from electrophysiological and imaging studies in the macaque monkey. Based on recent discovery of functionally specific domains in V4, we propose that the unifying function of V4 circuitry is to enable selective extraction of specific functional domain-based networks, whether it be by bottom-up specification of object features or by top-down attentionally driven selection.
A Neurophysiological and Computational Approach to the Functions of the Temporal Lobe Cortical Visual Areas in Invariant Object Recognition

Edmund T. Rolls

DOI: https://doi.org/10.1142/9789812799975_0035

2001-01-01

Vision

Abstract:Series on Biophysics and BiocyberneticsVision, pp. 366-395 (2001) No AccessFUNCTIONS OF THE PRIMATE TEMPORAL LOBE CORTICAL VISUAL AREAS IN INVARIANT VISUAL OBJECT AND FACE RECOGNITIONEDMUND T. ROLLSEDMUND T. ROLLSDepartment of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, Englandhttps://doi.org/10.1142/9789812799975_0035Cited by:1 PreviousNext AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack CitationsRecommend to Library ShareShare onFacebookTwitterLinked InRedditEmail Abstract: Neurophysiological evidence is described showing that some neurons in the macaque temporal cortical visual areas have responses that are invariant with respect to the position, size and view of faces and objects, and that these neurons show rapid processing and rapid learning. Which face or object is present is encoded using a distributed representation in which each neuron conveys independent information in its firing rate, with little information evident in the relative time of firing of neurons. This ensemble encoding has the advantages of maximising the information in the representation useful for discrimination between stimuli using a simple weighted sum of the neuronal firing by the receiving neurons, generalisation, and graceful degradation. In a clinical application of these findings, it is shown that humans with ventral frontal lobe damage have in some cases impairments in face and voice expression identification. These impairments are correlated with and may contribute to the problems some of these patients have in emotional and social behaviour. FiguresReferencesRelatedDetailsCited By 1Scale and translation-invariance for novel objects in human visionYena Han, Gemma Roig, Gad Geiger and Tomaso Poggio29 January 2020 | Scientific Reports, Vol. 10, No. 1 VisionMetrics History PDF download
A Neurodynamical Theory Of Visual Attention: Comparisons With Fmri- And Single-Neuron Data

G Deco,E Rolls

DOI: https://doi.org/10.1007/3-540-46084-5_1

2002-01-01

Abstract:We describe a model of invariant visual object recognition in the brain that incorporates different brain areas of the dorsal or 'where' and ventral or 'what' paths of the visual cortex. The dorsal 'where' path is implemented in the model by feedforward and feedback connections between brain areas V1, V2 and a PP module. The ventral 'what' path is implemented in a physiologically plausible four-layer network, corresponding to brain areas V1, V2, V4 and IT, with convergence to each part of a layer from a small region of the preceding layer, with feature-based attentional feedback connections, and with local competition between the neurons within a layer implemented by local lateral inhibition. In particular, the model explains the gradually increasing magnitude of the attentional modulation that is found in fMRI experiments from earlier visual areas (V1, V2) to higher ventral visual areas (V4, IT). The model also shows how the effective size of the receptive fields of IT neurons becomes smaller in natural cluttered scenes.
Outputs of visual processing

Edmund T. Rolls

DOI: https://doi.org/10.1093/acprof:oso/9780198524885.003.0012

2001-01-01

Abstract:Abstract This chapter examines the outputs of the ventral visual processing stream, from the inferior temporal visual cortical areas in particular. It explains that neuronal outputs provide a distributed representation of ‘what’ object or face is being viewed, while specialized subregions, such as the cortex in the superior temporal sulcus, provide evidence about face expression and about movements of objects and people. The chapter describes how attentional bias could be implemented by the operation of short-term-memory systems, explains why short-term memory must be separate from the temporal lobe and parietal lobe perceptual systems, and proposes a theory on how these short-term-memory systems work.
Invariant Visual Object and Face Recognition: Neural and Computational Bases, and a Model, VisNet

Edmund T Rolls

DOI: https://doi.org/10.3389/fncom.2012.00035

IF: 3.387

2012-01-01

Frontiers in Computational Neuroscience

Abstract:Neurophysiological evidence for invariant representations of objects and faces in the primate inferior temporal visual cortex is described. Then a computational approach to how invariant representations are formed in the brain is described that builds on the neurophysiology. A feature hierarchy model in which invariant representations can be built by self-organizing learning based on the temporal and spatial statistics of the visual input produced by objects as they transform in the world is described. VisNet can use temporal continuity in an associative synaptic learning rule with a short-term memory trace, and/or it can use spatial continuity in continuous spatial transformation learning which does not require a temporal trace. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in, for example, spatial and object search tasks. The approach has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene. The approach has also been extended to provide, with an additional layer, for the development of representations of spatial scenes of the type found in the hippocampus.
A Survey of Architecture and Function of the Primary Visual Cortex (V1)

Jeffrey Ng,Anil A. Bharath,Li Zhaoping

DOI: https://doi.org/10.1155/2007/97961

2006-01-01

EURASIP Journal on Advances in Signal Processing

Abstract:The largest visual area, known as the primary visual cortex or V1, has greatly contributed to the current understanding of mammalian and human visual pathways and their role in visual perception. The initial discovery of orientation-sensitive neurons in V1, arranged according to a retinotopic mapping, suggested an analogy to its function as a low-level feature analyzer. Subsequent discoveries of phase, spatial frequency, color, ocular origin, and direction-of-motion-sensitive neurons, arranged into overlapping maps, further lent support to the view that it performs a rich decomposition, similar to signal processing transforms, of the retinal output. Like the other cortical areas, V1 has a laminar organization with specialization for input from the relayed retinal afferents, output to the higher visual areas, and the segregation of the magno (motion) and parvo (form) pathways. Spatially lateral connections that exist between neurons of similar and varying properties have also been proposed to give rise to a computation of a bottom-up saliency map in V1. We provide a review of the selectivity of neurons in V1, laminar specialization and analogies to signal processing techniques, a model of V1 saliency computation, and higher-area feedback that may mediate perception.
Mesoscale functional organization and connectivity of color, disparity, and naturalistic texture in human second visual area

Hailin Ai,Weiru Lin,Chengwen Liu,Nihong Chen,Peng Zhang

DOI: https://doi.org/10.1101/2023.10.26.564178

2024-10-01

Abstract:Although parallel processing has been extensively studied in the low-level geniculostriate pathway and the high-level dorsal and ventral visual streams, less is known at the intermediate-level visual areas. In this study, we employed high-resolution fMRI at 7 Tesla to investigate the columnar and laminar organizations for color, disparity, and naturalistic texture in the human secondary visual cortex (V2), and their informational connectivity with lower and higher order visual areas. Although fMRI activations in V2 showed reproducible interdigitated color-selective thin and disparity-selective thick "stripe" columns, we found no clear evidence of columnar organization for naturalistic textures. Cortical depth-dependent analyses revealed the strongest color-selectivity in the superficial layers of V2, along with both feedforward and feedback informational connectivity with V1 and V4. Disparity selectivity was similar across different cortical depths of V2, which showed significant feedforward and feedback connectivity with V1 and V3ab. Interestingly, the selectivity for naturalistic texture was strongest in the deep layers of V2, with significant feedback connectivity from V4. Thus, while local circuitry within cortical columns is crucial for processing color and disparity information, feedback modulations from V4 play a dominant role in processing naturalistic textures in area V2.

Neuroscience
Face Processing in Different Brain Areas and Face Recognition

Edmund T Rolls

DOI: https://doi.org/10.1007/978-3-540-29678-2_1662

2017-01-01

Abstract:Face-selective neurons were discovered in the inferior temporal visual cortex by Perrett et al. (1982), in the amygdala by Sanghera, Rolls, and Roper-Hall (Sanghera et al. 1979; Rolls 2011) and Leonard et al. (1985), and in the orbitofrontal cortex by Rolls et al. (2006). These neurons respond 2–20 times more to the best face compared to the best nonface, and different brain regions provide the basis for face recognition and for emotional responses to faces. These neurons are described here.
Functional Organization for Color and Orientation in Macaque V4

Hisashi Tanigawa,Haidong D Lu,Anna W Roe

DOI: https://doi.org/10.1038/nn.2676

IF: 25

2010-01-01

Nature Neuroscience

Abstract:Visual area V4 in the macaque monkey is a cortical area that is strongly involved in color and shape perception. However, fundamental questions about V4 are still debated. V4 was initially characterized as a color-processing area, but subsequent studies revealed that it contains a diverse complement of cells, including those with preference for color, orientation, disparity and higher-order feature preferences. This has led to disputes and uncertainty about the role of V4 in vision. Using intrinsic signal optical imaging methods in awake, behaving monkeys, we found that different feature preferences are functionally organized in V4. Optical images revealed that regions with preferential response to color were largely separate from orientation-selective regions. Our results help to resolve long-standing controversies regarding functional diversity and retinotopy in V4 and indicate the presence of spatially biased distribution of featural representation in V4 in the ventral visual pathway.
Functional organization and natural scene responses across mouse visual cortical areas revealed with encoding manifolds

Luciano Dyballa,Greg D Field,Michael P Stryker,Steven W Zucker

DOI: https://doi.org/10.1101/2024.10.24.620089

2024-12-20

Abstract:A challenge in sensory neuroscience is understanding how populations of neurons operate in concert to represent diverse stimuli. To meet this challenge, we have created "encoding manifolds" that reveal the overall responses of brain areas to diverse stimuli with the resolution of individual neurons and their response dynamics. Here we use encoding manifold to compare the population-level encoding of primary visual cortex (VISp) with five higher visual areas (VISam, VISal, VISpm, VISlm, and VISrl). We used data from the Allen Institute Visual Coding–Neuropixels dataset from the mouse. We show that the encoding manifold topology computed only from responses to grating stimuli is continuous, for V1 and for higher visual areas, with smooth coordinates spanning it that include orientation selectivity and firing-rate magnitude. Surprisingly, the manifolds for each visual area revealed novel relationships between how natural scenes are encoded relative to static gratings—a relationship that was conserved across visual areas. Namely, neurons preferring natural scenes preferred either low or high spatial frequency gratings, but not intermediate ones. Analyzing responses by cortical layer reveals a preference for gratings concentrated in layer 6, whereas preferences for natural scenes tended to be higher in layers 2/3 and 4. The results reveal how machine learning approaches can be used to organize and visualize the structure of sensory coding, thereby revealing novel relationships within and across brain areas and sensory stimuli.

Neuroscience
Higher-Order Areas of the Mouse Visual Cortex

Lindsey L Glickfeld,Shawn R Olsen,Lindsey L. Glickfeld,Shawn R. Olsen

DOI: https://doi.org/10.1146/annurev-vision-102016-061331

IF: 7.745

2017-09-15

Annual Review of Vision Science

Abstract:The brain has evolved to transform sensory information in the environment into neural representations that can be used for perception and action. Higher-order sensory cortical areas, with their increasingly complex receptive fields and integrative properties, are thought to be critical nodes for this function. This is especially true in the primate visual cortex, in which functionally specialized areas are engaged in parallel streams to support diverse computations. Recent anatomical and physiological studies of the mouse visual cortex have revealed a similarly complex network of specialized higher-order areas. This structure provides a useful model for determining the synaptic and circuit mechanisms through which information is transformed across distinct processing stages. In this review, we summarize the current knowledge on the layout, connectivity, and functional properties of the higher visual areas in the mouse. In addition, we speculate on the contribution of these areas to perception and action, and how knowledge of the mouse visual system can inform us about the principles that govern information processing in integrated networks.

ophthalmology,neurosciences
Higher Order Visual Areas Enhance Stimulus Responsiveness in Mouse Primary Visual Cortex

Matthijs N Oude Lohuis,Alexis Cervan Canton,Cyriel M A Pennartz,Umberto Olcese

DOI: https://doi.org/10.1093/cercor/bhab414

IF: 4.861

2021-11-28

Cerebral Cortex

Abstract:Abstract Over the past few years, the various areas that surround the primary visual cortex (V1) in the mouse have been associated with many functions, ranging from higher order visual processing to decision-making. Recently, some studies have shown that higher order visual areas influence the activity of the primary visual cortex, refining its processing capabilities. Here, we studied how in vivo optogenetic inactivation of two higher order visual areas with different functional properties affects responses evoked by moving bars in the primary visual cortex. In contrast with the prevailing view, our results demonstrate that distinct higher order visual areas similarly modulate early visual processing. In particular, these areas enhance stimulus responsiveness in the primary visual cortex, by more strongly amplifying weaker compared with stronger sensory-evoked responses (for instance specifically amplifying responses to stimuli not moving along the direction preferred by individual neurons) and by facilitating responses to stimuli entering the receptive field of single neurons. Such enhancement, however, comes at the expense of orientation and direction selectivity, which increased when the selected higher order visual areas were inactivated. Thus, feedback from higher order visual areas selectively amplifies weak sensory-evoked V1 responses, which may enable more robust processing of visual stimuli.

neurosciences
The Neurophysiology and Computational Mechanisms of Object Representation

Edmund T. Rolls

DOI: https://doi.org/10.1017/cbo9780511635465.015

2009-01-01

Abstract:A concise description of the representation of objects and faces provided by inferior temporal cortex neurons in the primate (macaque) brain is followed by new findings about how this representation operates in natural scenes and allows a number of objects and their relative spatial position in a scene to be encoded. Then a computational approach to how the object recognition processes described are performed in the primate brain is discussed as well as the types of strategy that the human brain uses to solve the enormous computational problem of invariant object recognition in complex natural scenes (Rolls 2008b; Rolls and Deco 2002; Rolls and Stringer 2006b). This contribution aims to provide a closely linked neurophysiological and computational approach to object recognition and categorization. Other approaches are represented in this volume and elsewhere (Biederman 1987; Fukushima 1989; Riesenhuber and Poggio 2000; Serre et al. 2007), and are compared with current approaches (Rolls 2008b; Rolls and Deco 2002; Rolls and Stringer 2006b).
Visual brain and visual perception: how does the cortex do perceptual grouping?

Stephen Grossberg,Ennio Mingolla,William D Ross

DOI: https://doi.org/10.1016/s0166-2236(96)01002-8

1997-03-01

Trends in Neurosciences

Abstract:How the brain generates visual percepts is a central problem in neuroscience. We propose a detailed neural model of how lateral geniculate nuclei and the interblob cortical stream through V1 and V2 generate context-sensitive perceptual groupings from visual inputs. The model suggests a functional role for cortical layers, columns, maps and networks, and proposes homologous circuits for V1 and V2 with larger-scale processing in V2. An integrated treatment of interlaminar, horizontal, orientational and endstopping cortical interactions and a role for corticogeniculate feedback in grouping are proposed. Modeled circuits simulate parametric psychophysical data about boundary grouping and illusory contour formation.

neurosciences
Hierarchical organization of the human ventral visual streams revealed with magnetoencephalography

Edmund T Rolls,Gustavo Deco,Yi Zhang,Jianfeng Feng

DOI: https://doi.org/10.1093/cercor/bhad318

2023-10-09

Abstract:The hierarchical organization between 25 ventral stream visual cortical regions and 180 cortical regions was measured with magnetoencephalography using the Human Connectome Project Multimodal Parcellation atlas in 83 Human Connectome Project participants performing a visual memory task. The aim was to reveal the hierarchical organization using a whole-brain model based on generative effective connectivity with this fast neuroimaging method. V1-V4 formed a first group of interconnected regions. Especially V4 had connectivity to a ventrolateral visual stream: V8, the fusiform face cortex, and posterior inferior temporal cortex PIT. These regions in turn had effectivity connectivity to inferior temporal cortex visual regions TE2p and TE1p. TE2p and TE1p then have connectivity to anterior temporal lobe regions TE1a, TE1m, TE2a, and TGv, which are multimodal. In a ventromedial visual stream, V1-V4 connect to ventromedial regions VMV1-3 and VVC. VMV1-3 and VVC connect to the medial parahippocampal gyrus PHA1-3, which, with the VMV regions, include the parahippocampal scene area. The medial parahippocampal PHA1-3 regions have connectivity to the hippocampal system regions the perirhinal cortex, entorhinal cortex, and hippocampus. These effective connectivities of two ventral visual cortical streams measured with magnetoencephalography provide support to the hierarchical organization of brain systems measured with fMRI, and new evidence on directionality.
Inferior temporal cortical visual areas

Edmund T. Rolls

DOI: https://doi.org/10.1093/acprof:oso/9780198524885.003.0005

2001-01-01

Abstract:Abstract This chapter examines the role of the inferior temporal cortical visual areas in face recognition. It discusses neuronal responses found in different temporal lobe cortex visual areas, the selectivity of neurons for individual face features or for a combination of face features, and the distributed encoding of object and face identity. The chapter explains the representation of information in the responses of a population of inferior temporal visual cortex neurons and the temporal encoding in the spike train of a single neuron.
Computational neuroscience of vision

Edmund T. Rolls,Gustavo Deco

DOI: https://doi.org/10.1093/acprof:oso/9780198524885.001.0001

2002-01-01

Abstract:Abstract This book presents the highly complex subject of vision, focusing on the visual information processing and computational operations in the visual system that lead to representations of objects in the brain. In addition to visual processing, it also considers how visual inputs reach and are involved in the computations underlying a wide range of behaviour, thus providing a foundation for understanding the operation of a number of different brain systems.

Neural organization of higher visual functions

Vision, Emotion and Memory: from Neurophysiology to Computation

A mouse model of higher visual cortical function

A unifying framework for functional organization in early and higher ventral visual cortex

Toward a Unified Theory of Visual Area V4

A Neurophysiological and Computational Approach to the Functions of the Temporal Lobe Cortical Visual Areas in Invariant Object Recognition

A Neurodynamical Theory Of Visual Attention: Comparisons With Fmri- And Single-Neuron Data

Outputs of visual processing

Invariant Visual Object and Face Recognition: Neural and Computational Bases, and a Model, VisNet

A Survey of Architecture and Function of the Primary Visual Cortex (V1)

Mesoscale functional organization and connectivity of color, disparity, and naturalistic texture in human second visual area

Face Processing in Different Brain Areas and Face Recognition

Functional Organization for Color and Orientation in Macaque V4

Functional organization and natural scene responses across mouse visual cortical areas revealed with encoding manifolds

Higher-Order Areas of the Mouse Visual Cortex

Higher Order Visual Areas Enhance Stimulus Responsiveness in Mouse Primary Visual Cortex

The Neurophysiology and Computational Mechanisms of Object Representation

Visual brain and visual perception: how does the cortex do perceptual grouping?

Hierarchical organization of the human ventral visual streams revealed with magnetoencephalography

Inferior temporal cortical visual areas

Computational neuroscience of vision