One nose but two nostrils: Learn to align with sparse connections between two olfactory cortices

Bo Liu,Shanshan Qin,Venkatesh Murthy,Yuhai Tu
2024-05-07
Abstract:The integration of neural representations in the two hemispheres is an important problem in neuroscience. Recent experiments revealed that odor responses in cortical neurons driven by separate stimulation of the two nostrils are highly correlated. This bilateral alignment points to structured inter-hemispheric connections, but detailed mechanism remains unclear. Here, we hypothesized that continuous exposure to environmental odors shapes these projections and modeled it as online learning with local Hebbian rule. We found that Hebbian learning with sparse connections achieves bilateral alignment, exhibiting a linear trade-off between speed and accuracy. We identified an inverse scaling relationship between the number of cortical neurons and the inter-hemispheric projection density required for desired alignment accuracy, i.e., more cortical neurons allow sparser inter-hemispheric projections. We next compared the alignment performance of local Hebbian rule and the global stochastic-gradient-descent (SGD) learning for artificial neural networks. We found that although SGD leads to the same alignment accuracy with modestly sparser connectivity, the same inverse scaling relation holds. We showed that their similar performance originates from the fact that the update vectors of the two learning rules align significantly throughout the learning process. This insight may inspire efficient sparse local learning algorithms for more complex problems.
Neurons and Cognition,Biological Physics
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to achieve the alignment of neural representations in the bilateral olfactory cortices. Specifically, the researchers focus on how to achieve the consistency of bilateral olfactory - stimulus responses through sparse cross - hemispheric connections. This problem is very important in neuroscience because it involves the integration of information between the two hemispheres of the brain, which is crucial for forming a coherent perception of the external environment. The paper verifies the role of Hebbian learning rules in this alignment process through modeling and experimental verification, and explores the similarities and differences between local Hebbian learning and global stochastic gradient descent (SGD) learning in achieving this alignment effect. ### Main Findings 1. **Hebbian learning can achieve bilateral alignment**: Through sparse cross - hemispheric connections, the Hebbian learning rule can successfully achieve the alignment of neural representations in the bilateral olfactory cortices. The learning rate regulates the trade - off between alignment accuracy and convergence speed. 2. **Relationship between network size and connection density**: In order to reach a certain alignment level, the required cross - hemispheric connection density is inversely proportional to the number of cortical neurons. This means that as the number of cortical neurons increases, the number of cross - hemispheric connections required for each neuron can be reduced. 3. **Comparison between SGD and Hebbian learning**: Although SGD can achieve higher alignment accuracy in some cases, its performance tends to be the same as that of Hebbian learning in the case of high - input dimensions and dense feed - forward connections. This is because the weight update vectors of the two learning rules highly overlap under these conditions. ### Model and Training Process The researchers used a simplified neural network model to simulate the bilateral responses of the olfactory cortex. In the model, the responses \( \mathbf{x} \) of the olfactory bulb (OB) are transmitted to the bilateral olfactory cortices (OC), and different cortical representations \( \mathbf{r}_A \) and \( \mathbf{r}_B \) are formed through random connection matrices \( \mathbf{W}_A \) and \( \mathbf{W}_B \). The cross - hemispheric connection matrices \( \mathbf{G}_{AB} \) and \( \mathbf{G}_{BA} \) are updated through the Hebbian learning rule: \[ \Delta \mathbf{G}_{BA} = \eta (\mathbf{r}_B \mathbf{r}_A^T - \beta \mathbf{G}_{BA}) \odot \hat{\mathbf{G}}_{BA} \] where \( \eta \) is the learning rate, \( \beta \) is the weight decay constant, \( \odot \) represents element - wise multiplication, and \( \hat{\mathbf{G}}_{BA} \) is a fixed sparse mask matrix. ### Result Analysis 1. **Influence of learning rate**: A smaller learning rate \( \eta \) can improve alignment accuracy, but the convergence speed is slower. There is a linear trade - off between the learning rate and alignment accuracy and convergence speed. 2. **Influence of network size**: A larger network can achieve the same alignment performance through sparser connections. The required cross - hemispheric connection density is inversely proportional to the number of cortical neurons. 3. **Similarity between SGD and Hebbian learning**: Although SGD performs better in some cases, the weight update vectors of the two learning rules highly overlap under high - input dimensions and dense feed - forward connections, resulting in their similar final performance. ### Biological Significance These findings not only help to understand how the brain integrates bilateral olfactory information through sparse cross - hemispheric connections, but also provide a theoretical basis for designing efficient local learning algorithms. In particular, for large - scale neural networks, the Hebbian learning rule may be an effective choice because it can also achieve good alignment performance under sparse connections.