Abstract:Neural networks that can capture key principles underlying brain computation offer exciting new opportunities for developing artificial intelligence and brain-like computing algorithms. Such networks remain biologically plausible while leveraging localized forms of synaptic learning rules and modular network architecture found in the neocortex. Compared to backprop-driven deep learning approches, they provide more suitable models for deploying on neuromorphic hardware and have greater potential for scalability on large-scale computing clusters. The development of such brain-like neural networks depends on having a learning procedure that can build effective internal representations from data. In this work, we introduce and evaluate a brain-like neural network model capable of unsupervised representation learning. It builds on the Bayesian Confidence Propagation Neural Network (BCPNN), which has earlier been implemented as abstract as well as biophyscially detailed recurrent attractor neural networks explaining various cortical associative memory phenomena. Here we developed a feedforward BCPNN model to perform representation learning by incorporating a range of brain-like attributes derived from neocortical circuits such as cortical columns, divisive normalization, Hebbian synaptic plasticity, structural plasticity, sparse activity, and sparse patchy connectivity. The model was tested on a diverse set of popular machine learning benchmarks: grayscale images (MNIST, Fashion-MNIST), RGB natural images (SVHN, CIFAR-10), QSAR (MUV, HIV), and malware detection (EMBER). The performance of the model when using a linear classifier to predict the class labels fared competitively with conventional multi-layer perceptrons and other state-of-the-art brain-like neural networks.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is to develop a brain - like feed - forward neural network model capable of unsupervised representation learning. Specifically, the author hopes to introduce and evaluate a new model based on the Bayesian Confidence Propagation Neural Network (BCPNN), which can construct effective internal representations from data without relying on external labels. By combining brain - like properties in multiple cerebral cortex circuits (such as cortical columns, divisive normalization, Hebbian synaptic plasticity, structural plasticity, sparse activity, and sparse patch connections), this model aims to improve its performance in different machine - learning benchmark tests and compete with traditional multi - layer perceptrons and other state - of - the - art brain - like neural networks. ### Main problem summary: 1. **Unsupervised representation learning**: How to design a neural network that can automatically extract features from data without the need for external labels. 2. **Integration of brain - like properties**: How to incorporate multiple biological mechanisms in the cerebral cortex (such as Hebbian synaptic plasticity and structural plasticity) into artificial neural networks to improve their performance and biological plausibility. 3. **Practical application ability**: Whether the proposed model can perform well in multiple machine - learning tasks (such as image classification, QSAR prediction, and malware detection) and be as competitive as traditional deep - learning methods. ### Key improvement points: - **Structural plasticity algorithm**: Used to form sparse patch connections, which are formed under data - driven conditions. - **Continuous - value data interface**: Use the population vector coding method to convert continuous - value data features into a form that the BCPNN architecture can handle. - **Theoretical equivalence**: It is proved that the propagation and learning steps of the feed - forward BCPNN model are equivalent to the E - step and M - step of the expectation - maximization (EM) algorithm of the discrete mixture model. Through these improvements, the author hopes to show that this model not only theoretically conforms to the principles of brain - like computing but also can effectively perform unsupervised representation learning tasks in practice.

Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks

Spiking representation learning for associative memories

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Neocortical plasticity: an unsupervised cake but no free lunch

Convolutional Spike Timing Dependent Plasticity based Feature Learning in Spiking Neural Networks

Competitive learning to generate sparse representations for associative memory

Towards Biologically Plausible Deep Learning

A Unified Software/Hardware Scalable Architecture for Brain-Inspired Computing Based on Self-Organizing Neural Models

A Computational Model of Representation Learning in the Brain Cortex, Integrating Unsupervised and Reinforcement Learning

Emergence of Sparse Coding, Balance and Decorrelation from a Biologically-Grounded Spiking Neural Network Model of Learning in the Primary Visual Cortex

Semi-supervised learning with Bayesian Confidence Propagation Neural Network

A structural representing and learning model based on biological neural mechanism

Learning Sparse, Distributed Representations using the Hebbian Principle

Learning cortical hierarchies with temporal Hebbian updates

Memory-Dependent Computation and Learning in Spiking Neural Networks Through Hebbian Plasticity

Information bottleneck-based Hebbian learning rule naturally ties working memory and synaptic updates

Memory-enriched computation and learning in spiking neural networks through Hebbian plasticity

A Study of Biologically Plausible Neural Network: The Role and Interactions of Brain-Inspired Mechanisms in Continual Learning

Towards deep learning with spiking neurons in energy based models with contrastive Hebbian plasticity

Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learning