Abstract:The ability to stably maintain visual information over brief delays is central to cognitive functioning. One possible way to achieve robust working memory maintenance is by having multiple concurrent mnemonic representations across multiple cortical loci. For example, early visual cortex might contribute to storage by representing information in a 'sensory-like' format, while intraparietal sulcus uses a format transformed away from sensory driven responses. As an explicit test of mnemonic code transformations along the visual hierarchy, we quantitatively modeled the progression of veridical-to-categorical orientation representations in human participants. Participants directly viewed, or held in mind, an oriented grating pattern, and the similarity between fMRI activation patterns for different orientations was calculated throughout retinotopic cortex. During direct perception, similarity was clustered around cardinal orientations, while during working memory the obliques were represented more similarly. We modeled these similarity patterns based on the known distribution of orientation information in the natural world: The 'veridical' model uses an efficient coding framework to capture hypothesized representations during visual perception. The 'categorical' model assumes that different 'psychological distances' between orientations result in orientation categorization relative to cardinal axes. During direct perception, the veridical model explained the data well in early visual areas, while the categorical model did worse. During working memory, the veridical model only explained some of the data, while the categorical model gradually gained explanatory power for increasingly anterior retinotopic regions. These findings suggest that directly viewed images are represented veridically, but once visual information is no longer tethered to the sensory world, there is a gradual progression to more categorical mnemonic formats along the visual hierarchy.

Category Representation in Primary Visual Cortex after Visual Perceptual Learning

Perceptual learning and top-down influences in primary visual cortex.

Category Learning Selectively Enhances Representations of Boundary-Adjacent Exemplars in Early Visual Cortex

Perceptual Learning As a Result of Concerted Changes in Prefrontal and Visual Cortex

Learning to link visual contours.

Feedback from Human Posterior Parietal Cortex Enables Visuospatial Category Representations As Early As Primary Visual Cortex.

Learning to See: Experience and Attention in Primary Visual Cortex

Category learning increases discriminability of relevant object dimensions in visual cortex

Category learning can alter perception and its neural correlates

Mechanisms underlying category learning in the human ventral occipito-temporal cortex

Bottom-up saliency and top-down learning in the primary visual cortex of monkeys.

Perceptual Learning Enhanced the Coding Capacity of Primary Visual Cortex in Rhesus Monkey

Perceptual training continuously refines neuronal population codes in primary visual cortex

Dynamic categorization rules alter representations in human visual cortex

Neural Correlates of Category Learning in Monkey Inferior Temporal Cortex

Contour Saliency in Primary Visual Cortex

Newly Trained Lexical Categories Produce Lateralized Categorical Perception of Color.

A gradual transition toward categorical representations along the visual hierarchy during working memory, but not perception.

The Role of Top-Down Task Context in Learning to Perceive Objects

Categorical learning revealed in activity pattern of left fusiform cortex.

Mouse visual cortex areas represent perceptual and semantic features of learned visual categories