Voluntary control of semantic neural representations by imagery with conflicting visual stimulation

Ryohei Fukuma,Takufumi Yanagisawa,Shinji Nishimoto,Hidenori Sugano,Kentaro Tamura,Shota Yamamoto,Yasushi Iimura,Yuya Fujita,Satoru Oshino,Naoki Tani,Naoko Koide-Majima,Yukiyasu Kamitani,Haruhiko Kishima

DOI: https://doi.org/10.48550/arXiv.2112.01223

2021-11-07

Abstract:Neural representations of visual perception are affected by mental imagery and attention. Although attention is known to modulate neural representations, it is unknown how imagery changes neural representations when imagined and perceived images semantically conflict. We hypothesized that imagining an image would activate a neural representation during its perception even while watching a conflicting image. To test this hypothesis, we developed a closed-loop system to show images inferred from electrocorticograms using a visual semantic space. The successful control of the feedback images demonstrated that the semantic vector inferred from electrocorticograms became closer to the vector of the imagined category, even while watching images from different categories. Moreover, modulation of the inferred vectors by mental imagery depended on both the image category and time from the initiation of imagery. The closed-loop control of the semantic vectors revealed an asymmetrical interaction between visual perception and imagery.

Neurons and Cognition