Abstract:Backpropagation algorithm has been widely used as a mainstream learning procedure for neural networks in the past decade, and has played a significant role in the development of deep learning. However, there exist some limitations associated with this algorithm, such as getting stuck in local minima and experiencing vanishing/exploding gradients, which have led to questions about its biological plausibility. To address these limitations, alternative algorithms to backpropagation have been preliminarily explored, with the Forward-Forward (FF) algorithm being one of the most well-known. In this paper we propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples and thus leads to a more efficient process at both training and testing. Moreover, in our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems. The proposed method is evaluated on four public image classification benchmarks, and the experimental results illustrate significant improvement in prediction accuracy in comparison with the baseline.

A Fast Learning Algorithm for Feedfoward Neural Network Based on the Layer-by-Layer and Neuron-by-Neuron Optimizing Procedure

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

A Fast Learning Strategy For Multilayer Feedforward Neural Networks

A New Layer-Wise Linearized Algorithm for Feedforward Neural Network

Fast Learning Algorithms for Feedforward Neural Networks

Optimal Algorithm for a Multilayer Feedforward Net and Its Application

A Fast Learning Algorithm of Feedforward Neural Networks by Using Novel Error Functions

Adaptive Algorithm of Learning Rate for Feedforward Neural Network

A Fast Hybrid Algorithm Of Global Optimization For Feedforward Neural Networks

Speeding Up the Training Process of the Mfnn by Optimizing the Hidden Layers' Outputs

A new fast learning algorithm for feedforward neural networks using U-D factorization-based extended Kalman Filter*

A Fast Learning Algorithm of Neural Networks by Changing Error Functions

Least Square Learning Algorithm for Feed-Forward Neural Network

Optimal filtering algorithms for fast learning in feedforward neural networks

Train Feedfoward Neural Network with Layer-wise Adaptive Rate via Approximating Back-matching Propagation

A Fast Supervised Learning Algorithm of Multilayer Feedforward Network Based on Improved Conjugate Gradient Method

A Fast Learning Algorithm Of Neural Network For The Training And Recognition Of The Phonemes

Train feedfoward neural network with layer-wise adaptive rate via approximating back-matching propagation

The Cascaded Forward Algorithm for Neural Network Training

New Learning Algorithm for Feed-Forward Neural Network Based on Objective Back-Propagation

New Dynamical Optimal Learning for Linear Multilayer FNN