From Sensory to Perceptual Manifolds: The Twist of Neural Geometry

Heng Ma,Longsheng Jiang,Tao Liu,Jia Liu

DOI: https://doi.org/10.1101/2023.10.02.559721

2024-04-25

Abstract:To humans, nearly everything is classifiable: whether as big or small, edible or poisonous, righteous or unjust. Similarly, classification is a central task in many machine learning applications, yet the problem of linear inseparability has long posed challenges for artificial neural networks since their inception. Here we asked how biological neural networks tackle this problem by investigating the geometric embedding of neural manifolds in neural spaces of macaques’V2 during orientation discrimination of motion-induced illusory contours. Specifically, we constructed a three-dimensional stimulus space that inherently made the orientation classification of these contours a linearly inseparable problem. As expected, we identified a sensory manifold, formed by activities of the V2 neuron population, that faithfully corresponded to this stimulus space. Crucially, this sensory manifold underwent a series of twist operations, resulting in new axes orthogonal to the original ones. Within this expanded, high-dimensional perceptual manifold, the problem of linear inseparability became linearly separable. Computational models further revealed that the geometric twist operation was achieved by neurons exhibiting nonlinear mixed selectivity in networks with heterogeneous connectivity patterns. Taken together, our findings provide an example of how perception arises from sensation through the lens of neural geometry, enriching our understanding of how cognitive functions are derived from neuronal response profiles and their connectivity structure.

Neuroscience

What problem does this paper attempt to address?

The paper attempts to address the problem of how linear inseparable problems are processed in biological neural systems. Specifically, the researchers analyzed neuronal activity data from the V2 area of macaques to explore how the brain transforms an originally linear inseparable problem into a linear separable one when performing a direction classification task for motion-induced contours (MIC). The study found that through a series of "twist operations," the low-dimensional sensory manifold is converted into a high-dimensional perceptual manifold, in which the originally linear inseparable problem becomes linear separable. This process relies on the heterogeneous connection patterns of neurons with nonlinear mixed selectivity within the network. The findings provide a new perspective on how perception arises from sensory information and may offer insights for developing new machine learning algorithms.