Abstract:The human visual system uses two parallel pathways for spatial processing and object recognition. In contrast, computer vision systems tend to use a single feedforward pathway, rendering them less robust, adaptive, or efficient than human vision. To bridge this gap, we developed a dual-stream vision model inspired by the human eyes and brain. At the input level, the model samples two complementary visual patterns to mimic how the human eyes use magnocellular and parvocellular retinal ganglion cells to separate retinal inputs to the brain. At the backend, the model processes the separate input patterns through two branches of convolutional neural networks (CNN) to mimic how the human brain uses the dorsal and ventral cortical pathways for parallel visual processing. The first branch (WhereCNN) samples a global view to learn spatial attention and control eye movements. The second branch (WhatCNN) samples a local view to represent the object around the fixation. Over time, the two branches interact recurrently to build a scene representation from moving fixations. We compared this model with the human brains processing the same movie and evaluated their functional alignment by linear transformation. The WhereCNN and WhatCNN branches were found to differentially match the dorsal and ventral pathways of the visual cortex, respectively, primarily due to their different learning objectives. These model-based results lead us to speculate that the distinct responses and representations of the ventral and dorsal streams are more influenced by their distinct goals in visual attention and object recognition than by their specific bias or selectivity in retinal inputs. This dual-stream model takes a further step in brain-inspired computer vision, enabling parallel neural networks to actively explore and understand the visual surroundings.

Fusing Bottom-Up and Top-Down Pathways in Neural Networks for Visual Object Recognition

Object Recognition Using a Bio-Inspired Neuron Model with Bottom-Up and Top-Down Pathways

Vision at A Glance: Interplay between Fine and Coarse Information Processing Pathways

Integration of Bottom-Up and Top-Down Cues in Bayesian Network for Object Detection

BEYOND SUPERVISED LEARNING: RECOGNIZING UNSEEN ATTRIBUTE OBJECT PAIRS WITH VISION LANGUAGE FUSION AND ATTRACTOR NETWORKS

Visual information processing through the interplay between fine and coarse signal pathways

Plan-Recognition-Driven Attention Modeling for Visual Recognition

A brain-inspired object-based attention network for multiobject recognition and visual reasoning

Basic Level Categorization Facilitates Visual Object Recognition

Selective Segmentation Networks Using Top-Down Attention

Multiple networks fusion approach for 3-D object recognition from 2-D view

Learning Siamese Network with Top-Down Modulation for Visual Tracking

A Dual-Stream Neural Network Explains the Functional Segregation of Dorsal and Ventral Visual Pathways in Human Brains

Top-Down Processing: Top-Down Network Combines Back-Propagation with Attention

Visual Attentional Network and Learning Method for Object Search and Recognition

An Iterative and Cooperative Top-Down and Bottom-Up Inference Network for Salient Object Detection.

Exploring neural architectures for simultaneously recognizing multiple visual attributes

Brain-inspired Dual-Pathway Neural Network Architecture and Its Generalization Analysis

The attentive reconstruction of objects facilitates robust object recognition

Ventral-Dorsal Neural Networks: Object Detection via Selective Attention

Model of Human Visual Cortex Inspired Computational Models for Visual Recognition