Abstract:Humans are extremely robust in our ability to perceive and recognize objects—we see faces in tea stains and can recognize friends on dark streets. Yet, neurocomputational models of primate object recognition have focused on the initial feed-forward pass of processing through the ventral stream and less on the top-down feedback that likely underlies robust object perception and recognition. Aligned with the generative approach, we propose that the visual system actively facilitates recognition by reconstructing the object hypothesized to be in the image. Top-down attention then uses this reconstruction as a template to bias feedforward processing to align with the most plausible object hypothesis. Building on auto-encoder neural networks, our model makes detailed hypotheses about the appearance and location of the candidate objects in the image by reconstructing a complete object representation from potentially incomplete visual input due to noise and occlusion. The model then leverages the best object reconstruction, measured by reconstruction error, to direct the bottom-up process of selectively routing low-level features, a top-down biasing that captures a core function of attention. We evaluated our model using the MNIST-C (handwritten digits under corruptions) and ImageNet-C (real-world objects under corruptions) datasets. Not only did our model achieve superior performance on these challenging tasks designed to approximate real-world noise and occlusion viewing conditions, but also better accounted for human behavioral reaction times and error patterns than a standard feedforward Convolutional Neural Network. Our model suggests that a complete understanding of object perception and recognition requires integrating top-down and attention feedback, which we propose is an object reconstruction. Humans can dream and imagine things, and this means that the human brain can generate perceptions of things that are not there. We propose that humans evolved this generation capability, not solely to have more vivid dreams, but to help us better understand the world, especially when what we see is unclear or missing some details (due to occlusion, changing perspective, etc.). Through a combination of computational modeling and behavioral experiments, we demonstrate how the process of generating objects—actively reconstructing the most plausible object representation from noisy visual input—guides attention towards specific features or locations within an image (known as functions of top-down attention), thereby enhancing the system's robustness to various types of noise and corruption. We found that this generative attention mechanism could explain, not only the time that it took people to recognize challenging objects, but also the types of recognition errors made by people (seeing an object as one thing when it was really another). These findings contribute to a deeper understanding of the computational mechanisms of attention in the brain and their potential connection to the generative processes that facilitate robust object recognition.

The attentive reconstruction of objects facilitates robust object recognition

The attentive reconstruction of objects facilitates robust object recognition

Reconstruction-guided attention improves the robustness and shape processing of neural networks

A brain-inspired object-based attention network for multiobject recognition and visual reasoning

Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation

Object Reconstruction Based on Attentive Recurrent Network from Single and Multiple Images

Top-down attention based on object representation and incremental memory for knowledge building and inference

Configural processing as an optimized strategy for robust object recognition in neural networks

The Quest for an Integrated Set of Neural Mechanisms Underlying Object Recognition in Primates

TDAPNet: Prototype Network with Recurrent Top-Down Attention for Robust Object Classification under Partial Occlusion

Human Eyes Inspired Recurrent Neural Networks are More Robust Against Adversarial Noises

Understanding top-down attention using task-oriented ablation design

Rethinking Attentive Object Detection Via Neural Attention Learning.

Human Eyes-Inspired Recurrent Neural Networks Are More Robust Against Adversarial Noises

Contact-conditioned Hand-Held Object Reconstruction from Single-View Images

Leveraging the Human Ventral Visual Stream to Improve Neural Network Robustness

Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition

Improved object recognition using neural networks trained to mimic the brain's statistical properties

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

A Neurodynamical Cortical Model of Visual Attention and Invariant Object Recognition

Hand-object reconstruction via interaction-aware graph attention mechanism