Sensory Input Matching Visual Working Memory Guides Internal Prioritization

Damian Koevoet,Christoph Strauch,Marnix Naber,Stefan Van der Stigchel

DOI: https://doi.org/10.1101/2024.02.07.579262

2024-05-28

Abstract:Adaptive behavior necessitates the prioritization of the most relevant information in the environment (external) and in memory (internal). Internal prioritization is known to guide the selection of external sensory input, but the reverse may also be possible: Does the environment guide the prioritization of memorized material? Here we addressed whether reappearing sensory input matching visual working memory (VWM) facilitates the prioritization of other non-reappearing memorized items. Participants (total n = 72) memorized three orientations. Crucially some, but not all, items maintained in VWM were made available again in the environment. These reappearing items never had to be reproduced later. Experiment 1 showed that the reappearance of all but one memory item benefited accuracy and speed to the same extent as a spatial retro cue. This shows that reappearing memory-matching items allow for the dynamic prioritization of another non-reappearing memorized item. What aspects of the reappearing sensory input drive this effect? Experiment 2 demonstrated that prioritization was facilitated most if reappearing items matched VWM content in terms of both location and orientation. Sensory input fully matching VWM is likely processed more efficiently, possibly leading to stronger prioritization of other memory content. We showed the robustness of our findings in Experiment 3. We propose that the link between sensory processing and VWM is bidirectional: internal representations guide the processing of sensory input, which in turn facilitates the prioritization of other VWM content to subserve adaptive behavior.

Neuroscience

What problem does this paper attempt to address?

### The Problem the Paper Attempts to Solve

This paper aims to explore whether sensory input can guide the prioritization of visual working memory (VWM) content. Specifically, the researchers want to understand if the reappearance of sensory input that matches memory items in VWM can facilitate the prioritization of other non-reappearing memory items.

### Background and Motivation

In daily life, objects in the environment continuously appear, disappear, and reappear, presenting a challenge to our attention and memory systems with this dynamic flow of information. Traditional research has mainly focused on the prioritization of VWM content under static display conditions, which do not fully reflect the dynamic situations of the real world. Therefore, this paper uses experimental design to simulate how the reappearance of objects in a dynamic environment affects the prioritization of VWM.

### Experimental Design and Results

#### Experiment 1
- **Method**: Participants were required to remember three Gabor patches with different orientations. After a delay period, some memory items reappeared in the environment, but these reappearing items were not tested at the end of the trial.
- **Results**: When two memory items reappeared, participants' accuracy and reaction speed in reporting the remaining items significantly improved, similar to the effect of spatial retro-cues. This suggests that reappearing items can dynamically help prioritize other non-reappearing memory items.

#### Experiment 2
- **Method**: Further explored the impact of the location and orientation information of reappearing items on VWM prioritization. The experiment included four conditions: combination (both location and orientation match), location (only location matches), orientation (only orientation matches), and control (no information cue).
- **Results**: When the reappearing items matched in both location and orientation, participants' accuracy and reaction speed were the highest. Location information was more effective than orientation information in promoting VWM prioritization.

#### Experiment 3
- **Method**: To verify the results of the first two experiments and control for low-level visual adaptation effects, a mask was briefly presented after the encoding phase. All conditions included a black square probe.
- **Results**: Once again confirmed that performance in the combination condition was better than in the control condition, and location information had a more significant effect on promoting VWM prioritization.

### Conclusion

The study found that when reappearing sensory input matches memory items in VWM, it can effectively facilitate the prioritization of other non-reappearing memory items. This facilitation effect is most pronounced when the reappearing items match in both location and orientation. These results provide new insights into the mechanisms of VWM operation in dynamic environments.