Abstract:Training recurrent neural networks typically relies on backpropagation through time (BPTT). BPTT depends on forward and backward passes to be completed, rendering the network locked to these computations before loss gradients are available. Recently, Jaderberg et al. proposed synthetic gradients to alleviate the need for full BPTT. In their implementation synthetic gradients are learned through a mixture of backpropagated gradients and bootstrapped synthetic gradients, analogous to the temporal difference (TD) algorithm in Reinforcement Learning (RL). However, as in TD learning, heavy use of bootstrapping can result in bias which leads to poor synthetic gradient estimates. Inspired by the accumulate $\mathrm{TD}(\lambda)$ in RL, we propose a fully online method for learning synthetic gradients which avoids the use of BPTT altogether: accumulate $BP(\lambda)$. As in accumulate $\mathrm{TD}(\lambda)$, we show analytically that accumulate $\mathrm{BP}(\lambda)$ can control the level of bias by using a mixture of temporal difference errors and recursively defined eligibility traces. We next demonstrate empirically that our model outperforms the original implementation for learning synthetic gradients in a variety of tasks, and is particularly suited for capturing longer timescales. Finally, building on recent work we reflect on accumulate $\mathrm{BP}(\lambda)$ as a principle for learning in biological circuits. In summary, inspired by RL principles we introduce an algorithm capable of bias-free online learning via synthetic gradients.

BG: a hybrid algorithm for MLP

Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and Bp Algorithms

Genetic Algorithms for MLP Neural Network parameters optimization

Fast Algorithm for Adaptive Generalized Predictive Control Based on Bp Neural Networks

An optimizing BP neural network algorithm based on genetic algorithm

A Rbf Network Modeling Approach Combining Ga and Orthogonal Optimum Seeking Method

A hybrid training algorithm based on gradient descent and evolutionary computation

Two Hybrid Learning Strategies of Feedforward Network

A Fast Hybrid Algorithm Of Global Optimization For Feedforward Neural Networks

Extended Mixture of MLP Experts by Hybrid of Conjugate Gradient Method and Modified Cuckoo Search

Optimization Study of BP Neural Network Based on Genetic Algorithm

BP(λ): Online Learning via Synthetic Gradients

Using Genetic Algorithm to Optimize BP Neural Network

Fast Learning Algorithms for Feedforward Neural Networks

A Hybrid Variable Selection Algorithm for Multi-Layer Perceptron with Nonnegative Garrote and Extremal Optimization

Multistrategy Learning Using Genetic Algorithms and Neural Networks for Pattern Classification

Brief Introduction of Back Propagation (BP) Neural Network Algorithm and Its Improvement

The Combination Of Grey System And Bp Neural Network

An Integrated Hybrid Algorithm Based On Nature Inspired Evolutionary For Radial Basis Function Neural Network Learning

Heterogeneous Multilayer Generalized Operational Perceptron

Study on Learning Algorithm of a Kind of Multi-Neural Networks Hybrid Model