Abstract:Object recognition requires flexible and robust information processing, especially in view of the challenges posed by naturalistic visual settings. The ventral stream in visual cortex is provided with this robustness by its recurrent connectivity. Recurrent deep neural networks (DNNs) have recently emerged as promising models of the ventral stream. In this study, we asked whether DNNs could be used to explore the role of different recurrent computations during challenging visual recognition. We assembled a stimulus set that included manipulations that are often associated with recurrent processing in the literature, like occlusion, partial viewing, clutter, and spatial phase scrambling. We obtained a benchmark dataset from human participants performing a categorisation task on this stimulus set. By applying a wide range of model architectures to the same task, we uncovered a nuanced relationship between recurrence, model size, and performance. While recurrent models reach higher performance than their feedforward counterpart, we could not dissociate this improvement from that obtained by increasing model size. We found consistency between humans and models patterns of difficulty across the visual manipulations, but this was not modulated in an obvious way by the specific type of recurrence or size added to the model. Finally, depth/size rather than recurrence makes model confusion patterns more human-like. Contrary to previous assumptions, our findings challenge the notion that recurrent models are better models of human recognition behaviour than feedforward models, and emphasise the complexity of incorporating recurrence into computational models.

Recurrent Models of Visual Attention

RECL: Responsive Resource-Efficient Continuous Learning for Video Analytics

Long-Term Recurrent Convolutional Networks for Visual Recognition and Description

Recurrent convolutional video captioning with global and local attention.

Task-Driven Convolutional Recurrent Models of the Visual System

General Recurrent Attention Model for Jointly Multiple Object Recognition and Weakly Supervised Localization.

Where to Look: A Unified Attention Model for Visual Recognition with Reinforcement Learning

Recurrent issues with deep neural networks of visual recognition

Recurrent Joint Embedding Predictive Architecture with Recurrent Forward Propagation Learning

Recurrent Filter Learning for Visual Tracking

Recurrent 3D Attentional Networks for End-to-end Active Object Recognition

Human Eyes Inspired Recurrent Neural Networks are More Robust Against Adversarial Noises

RATM: Recurrent Attentive Tracking Model

Recurrent networks improve neural response prediction and provide insights into underlying cortical circuits

Human Eyes-Inspired Recurrent Neural Networks Are More Robust Against Adversarial Noises

End-to-End Instance Segmentation with Recurrent Attention

Recurrent Attentional Networks for Saliency Detection

Learning Wake-Sleep Recurrent Attention Models

Better Deep Visual Attention with Reinforcement Learning in Action Recognition.

Recurrent network dynamics reconciles visual motion segmentation and integration

Learning what to look in chest X-rays with a recurrent visual attention model