Abstract:This paper presents a new learning algorithm, termed Deep Bi-directional Predictive Coding (DBPC) that allows developing networks to simultaneously perform classification and reconstruction tasks using the same weights. Predictive Coding (PC) has emerged as a prominent theory underlying information processing in the brain. The general concept for learning in PC is that each layer learns to predict the activities of neurons in the previous layer which enables local computation of error and in-parallel learning across layers. In this paper, we extend existing PC approaches by developing a network which supports both feedforward and feedback propagation of information. Each layer in the networks trained using DBPC learn to predict the activities of neurons in the previous and next layer which allows the network to simultaneously perform classification and reconstruction tasks using feedforward and feedback propagation, respectively. DBPC also relies on locally available information for learning, thus enabling in-parallel learning across all layers in the network. The proposed approach has been developed for training both, fully connected networks and convolutional neural networks. The performance of DBPC has been evaluated on both, classification and reconstruction tasks using the MNIST and FashionMNIST datasets. The classification and the reconstruction performance of networks trained using DBPC is similar to other approaches used for comparison but DBPC uses a significantly smaller network. Further, the significant benefit of DBPC is its ability to achieve this performance using locally available information and in-parallel learning mechanisms which results in an efficient training protocol. This results clearly indicate that DBPC is a much more efficient approach for developing networks that can simultaneously perform both classification and reconstruction.

Multi-back-propagation Algorithm for Signal Neural Network Decomposition

Automatic Feature Extraction Based Structure Decomposition Method for Multi-Classification

Desire Backpropagation: A Lightweight Training Algorithm for Multi-Layer Spiking Neural Networks based on Spike-Timing-Dependent Plasticity

Dendritic cortical microcircuits approximate the backpropagation algorithm

Spike time displacement based error backpropagation in convolutional spiking neural networks

Learning multi‐modal recurrent neural networks with target propagation

Single-phase deep learning in cortico-cortical networks

Learning representations by back-propagating errors

Supervised Learning in Multilayer Spiking Neural Networks With Spike Temporal Error Backpropagation

Memorized Sparse Backpropagation.

RRCNN: A novel signal decomposition approach based on recurrent residue convolutional neural network

Spatiotemporal Backpropagation based on Channel Reward for Training High-Precision Spiking Neural Network

Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry

Approximating Back-propagation for a Biologically Plausible Local Learning Rule in Spiking Neural Networks

Learning efficient backprojections across cortical hierarchies in real time

Error Forward-Propagation: Reusing Feedforward Connections to Propagate Errors in Deep Learning

The Integrated Forward-Forward Algorithm: Integrating Forward-Forward and Shallow Backpropagation With Local Losses

A Spiking Network for Inference of Relations Trained with Neuromorphic Backpropagation

Multimodal Moore-Penrose Inverse-Based Recomputation Framework for Big Data Analysis

Deep Predictive Coding with Bi-directional Propagation for Classification and Reconstruction

Dendritic error backpropagation in deep cortical microcircuits