Abstract:Abstract The two visual cortical stream hypothesis, which suggests object properties (what) are processed separately from spatial properties (where), has a longstanding history, and much evidence has accumulated to support its conjectures. Nevertheless, in the last few decades, conflicting evidence has mounted that demands some explanation and modification. For example, existence of (1) shape activities (fMRI) or shape selectivities (physiology) in dorsal stream, similar to ventral stream; likewise, spatial activations (fMRI) or spatial selectivities (physiology) in ventral stream, similar to dorsal stream; (2) multiple segregated subpathways within a stream. In addition, the idea of segregation of various aspects of multiple objects in a scene raises questions about how these properties of multiple objects are then properly re-associated or bound back together to accurately perceive, remember, or make decisions. We will briefly review the history of the two-stream hypothesis, discuss competing accounts that challenge current thinking, and propose ideas on why the brain has segregated pathways. We will present ideas based on our own data using artificial neural networks (1) to reveal encoding differences for what and where that arise in a two-pathway neural network, (2) to show how these encoding differences can clarify previous conflicting findings, and (3) to elucidate the computational advantages of segregated pathways. Furthermore, we will discuss whether neural networks need to have multiple subpathways for different visual attributes. We will also discuss the binding problem (how to correctly associate the different attributes of each object together when there are multiple objects each with multiple attributes in a scene) and possible solutions to the binding problem. Finally, we will briefly discuss problems and limitations with existing models and potential fruitful future directions.

Multiple Cortical Visual Streams in Humans

Two What, Two Where, Visual Cortical Streams in Humans

Hierarchical organization of the human ventral visual streams revealed with magnetoencephalography

Hierarchical organisation of the human ventral visual streams revealed with magnetoencephalography

Selective activations and functional connectivities to the sight of faces, scenes, body parts and tools in visual and non-visual cortical regions leading to the human hippocampus

A ventromedial visual cortical 'Where' stream to the human hippocampus for spatial scenes revealed with magnetoencephalography

Understanding Cortical Streams from a Computational Perspective

Auditory, Visual and Audiovisual Speech Processing Streams in Superior Temporal Sulcus

Auditory Cortical Connectivity in Humans

Prefrontal and Somatosensory-Motor Cortex Effective Connectivity in Humans

A single computational objective drives specialization of streams in visual cortex

Extensive Cortical Connectivity of the Human Hippocampal Memory System: Beyond the “what” and “where” Dual Stream Model

Two hierarchically organized neural systems for object information in human visual cortex

Spontaneous Activity in the Visual Cortex is Organized by Visual Streams

Parallel Streams of Direct Corticogeniculate Feedback from Mid-level Extrastriate Cortex in the Macaque Monkey

Vision, Emotion and Memory: from Neurophysiology to Computation

Neural substrate for the engagement of the ventral visual stream in motor control in the macaque monkey

Finding and Recognizing Objects in Natural Scenes: Complementary Computations in the Dorsal and Ventral Visual Systems

Combined diffusion-weighted and functional magnetic resonance imaging reveals a temporal-occipital network involved in auditory-visual object processing

Distributed representation of objects in the human ventral visual pathway

Mesoscale functional organization and connectivity of color, disparity, and naturalistic texture in human second visual area