Abstract:Neural networks that can capture key principles underlying brain computation offer exciting new opportunities for developing artificial intelligence and brain-like computing algorithms. Such networks remain biologically plausible while leveraging localized forms of synaptic learning rules and modular network architecture found in the neocortex. Compared to backprop-driven deep learning approches, they provide more suitable models for deploying on neuromorphic hardware and have greater potential for scalability on large-scale computing clusters. The development of such brain-like neural networks depends on having a learning procedure that can build effective internal representations from data. In this work, we introduce and evaluate a brain-like neural network model capable of unsupervised representation learning. It builds on the Bayesian Confidence Propagation Neural Network (BCPNN), which has earlier been implemented as abstract as well as biophyscially detailed recurrent attractor neural networks explaining various cortical associative memory phenomena. Here we developed a feedforward BCPNN model to perform representation learning by incorporating a range of brain-like attributes derived from neocortical circuits such as cortical columns, divisive normalization, Hebbian synaptic plasticity, structural plasticity, sparse activity, and sparse patchy connectivity. The model was tested on a diverse set of popular machine learning benchmarks: grayscale images (MNIST, Fashion-MNIST), RGB natural images (SVHN, CIFAR-10), QSAR (MUV, HIV), and malware detection (EMBER). The performance of the model when using a linear classifier to predict the class labels fared competitively with conventional multi-layer perceptrons and other state-of-the-art brain-like neural networks.

Semi-supervised learning with Bayesian Confidence Propagation Neural Network

Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks

Locally Weighted Learning: How and when Does It Work in Bayesian Networks?

Semi-supervised Dynamic Counter Propagation Network

Belief Propagation Network for Hard Inductive Semi-Supervised Learning

Semi-supervised deep learning based on label propagation in a 2D embedded space

Semi-supervised Learning for K-Dependence Bayesian Classifiers

Bayesian Graph Convolutional Neural Networks for Semi-Supervised Classification

Learning Bayesian Multinets from Labeled and Unlabeled Data for Knowledge Representation.

Incorporating Unlabelled Data into Bayesian Neural Networks

Bayesian Semi-supervised Multi-category Classification under Nonparanormality

Semi-supervised Learning with Adaptive Neighborhood Graph Propagation Network

Semi-Supervised Classification Using Linear Neighborhood Propagation

Semi-Supervised Image Classification by Nonnegative Sparse Neighborhood Propagation

Learning Hierarchical Representations for Face Recognition Using Deep Belief Network Embedded with Softmax Regress and Multiple Neural Networks

Semi-Supervised Learning Classifier System Based on Bayes

Attention-based Graph Neural Network for Semi-supervised Learning

Semi-Supervised Neuron Segmentation via Reinforced Consistency Learning

A Dynamic Bayesian Networks Algorithm Based On Semi-Supervised Active Learning

Semi-supervised Bayesian Network Classifier Learning Based on Inter-Relation Mining among Attributes

A Dynamic Bayesian Network Algorithm Based on Semi-Supervised Active Learning