Abstract:Neural networks that can capture key principles underlying brain computation offer exciting new opportunities for developing artificial intelligence and brain-like computing algorithms. Such networks remain biologically plausible while leveraging localized forms of synaptic learning rules and modular network architecture found in the neocortex. Compared to backprop-driven deep learning approches, they provide more suitable models for deploying on neuromorphic hardware and have greater potential for scalability on large-scale computing clusters. The development of such brain-like neural networks depends on having a learning procedure that can build effective internal representations from data. In this work, we introduce and evaluate a brain-like neural network model capable of unsupervised representation learning. It builds on the Bayesian Confidence Propagation Neural Network (BCPNN), which has earlier been implemented as abstract as well as biophyscially detailed recurrent attractor neural networks explaining various cortical associative memory phenomena. Here we developed a feedforward BCPNN model to perform representation learning by incorporating a range of brain-like attributes derived from neocortical circuits such as cortical columns, divisive normalization, Hebbian synaptic plasticity, structural plasticity, sparse activity, and sparse patchy connectivity. The model was tested on a diverse set of popular machine learning benchmarks: grayscale images (MNIST, Fashion-MNIST), RGB natural images (SVHN, CIFAR-10), QSAR (MUV, HIV), and malware detection (EMBER). The performance of the model when using a linear classifier to predict the class labels fared competitively with conventional multi-layer perceptrons and other state-of-the-art brain-like neural networks.

Deep Associative Learning for Neural Networks

A neuron-weighted learning algorithm and its hardware implementation in associative memories

Deep Associative Neural Network for Associative Memory Based on Unsupervised Representation Learning

Learning Paired-associate Images with An Unsupervised Deep Learning Architecture

Neural auto-association with optimal Bayesian learning

Associative Memory Optimized Method on Deep Neural Networks for Image Classification.

Probabilistic Reasoning Via Deep Learning: Neural Association Models

Bridging Associative Memory and Probabilistic Modeling

Learning Sparse, Distributed Representations using the Hebbian Principle

Evolving Connections in Group of Neurons for Robust Learning

Associative Learning for Network Embedding

A learning algorithm of multilaver dynamics associative neural network based on generalized hebb rule

Learning in Associative Networks through Pavlovian Dynamics

Towards Biologically Plausible Deep Learning

A Novel Neural Network for Associative Memory Via Dynamical Systems

Learning data association without data association: An EM approach to neural assignment prediction

Dense Associative Memory for Pattern Recognition

Biology-inspired joint distribution neurons based on Hierarchical Correlation Reconstruction allowing for multidirectional neural networks

Continual Learning with Deep Artificial Neurons

Efficient learning algorithm for associative memory neural network

Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks