Perceptography unveils the causal contribution of inferior temporal cortex to visual perception

Elia Shahbazi,Timothy Ma,Martin Pernuš,Walter Scheirer,Arash Afraz
DOI: https://doi.org/10.1038/s41467-024-47356-8
IF: 16.6
2024-04-18
Nature Communications
Abstract:Abstract Neurons in the inferotemporal (IT) cortex respond selectively to complex visual features, implying their role in object perception. However, perception is subjective and cannot be read out from neural responses; thus, bridging the causal gap between neural activity and perception demands independent characterization of perception. Historically, though, the complexity of the perceptual alterations induced by artificial stimulation of IT cortex has rendered them impossible to quantify. To address this old problem, we tasked male macaque monkeys to detect and report optical impulses delivered to their IT cortex. Combining machine learning with high-throughput behavioral optogenetics, we generated complex and highly specific images that were hard for the animal to distinguish from the state of being cortically stimulated. These images, named “perceptograms” for the first time, reveal and depict the contents of the complex hallucinatory percepts induced by local neural perturbation in IT cortex. Furthermore, we found that the nature and magnitude of these hallucinations highly depend on concurrent visual input, stimulation location, and intensity. Objective characterization of stimulation-induced perceptual events opens the door to developing a mechanistic theory of visual perception. Further, it enables us to make better visual prosthetic devices and gain a greater understanding of visual hallucinations in mental disorders.
multidisciplinary sciences
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to objectively quantify the perceptual changes caused by artificial stimulation of specific regions of the brain, especially the inferior temporal lobe cortex (IT cortex). Specifically, the authors trained male macaques to detect and report the optical pulse stimulation of their IT cortex by combining machine learning with high - throughput behavioral optogenetics techniques. This research aims to fill the causal gap between neural activity and perception, thereby providing a new perspective for understanding the mechanisms of visual perception and laying the foundation for further development of visual prosthetic devices and understanding of visual hallucinations in mental disorders. The key challenge in the study is that although artificial stimulation of high - level visual cortex regions can trigger hallucinatory perceptions, the complexity and subjectivity of these perceptions make them difficult to be scientifically characterized. The paper introduces a new method named "perceptography", which can generate complex and highly specific images (called "perceptograms"). These images can reveal and depict the content of complex hallucinatory perceptions induced by local neural perturbations in the IT cortex. This method can not only provide an objective representation of stimulation - induced perceptual events, but also explore how these hallucinations are affected by current visual input, stimulation location and intensity. Through this research, the authors hope to open the door to establishing a mechanical theory of visual perception, promote the development of better visual prosthetic devices, and deepen the understanding of visual hallucinations in mental disorders.