Abstract:This paper presents a new learning algorithm, termed Deep Bi-directional Predictive Coding (DBPC) that allows developing networks to simultaneously perform classification and reconstruction tasks using the same weights. Predictive Coding (PC) has emerged as a prominent theory underlying information processing in the brain. The general concept for learning in PC is that each layer learns to predict the activities of neurons in the previous layer which enables local computation of error and in-parallel learning across layers. In this paper, we extend existing PC approaches by developing a network which supports both feedforward and feedback propagation of information. Each layer in the networks trained using DBPC learn to predict the activities of neurons in the previous and next layer which allows the network to simultaneously perform classification and reconstruction tasks using feedforward and feedback propagation, respectively. DBPC also relies on locally available information for learning, thus enabling in-parallel learning across all layers in the network. The proposed approach has been developed for training both, fully connected networks and convolutional neural networks. The performance of DBPC has been evaluated on both, classification and reconstruction tasks using the MNIST and FashionMNIST datasets. The classification and the reconstruction performance of networks trained using DBPC is similar to other approaches used for comparison but DBPC uses a significantly smaller network. Further, the significant benefit of DBPC is its ability to achieve this performance using locally available information and in-parallel learning mechanisms which results in an efficient training protocol. This results clearly indicate that DBPC is a much more efficient approach for developing networks that can simultaneously perform both classification and reconstruction.

Deep Predictive Coding with Bi-directional Propagation for Classification and Reconstruction

Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?

A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks

Predictive Coding as a Neuromorphic Alternative to Backpropagation: A Critical Evaluation

Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networks

Understanding Predictive Coding as an Adaptive Trust-Region Method

Classification and Reconstruction Processes in Deep Predictive Coding Networks: Antagonists or Allies?

Predictive Coding beyond Gaussian Distributions

Reverse Differentiation via Predictive Coding

Predictive coding with spiking neurons and feedforward gist signaling

Fast Inference Predictive Coding: A Novel Model for Constructing Deep Neural Networks

Nonlinear Prediction of Multidimensional Signals Via Deep Regression with Applications to Image Coding.

Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks

Sparse deep predictive coding captures contour integration capabilities of the early visual system

Hierarchical Discriminative Sparse Coding Via Bidirectional Connections

Learning on Arbitrary Graph Topologies via Predictive Coding

Understanding and Improving Optimization in Predictive Coding Networks

A Predictive-Coding Network That Is Both Discriminative and Generative

Deep Convolutional Network Based on Pyramid Architecture

Hierarchical Predictive Coding Models in a Deep-Learning Framework