Abstract:An ongoing challenge in neural information processing is the following question: how do neurons adjust their connectivity to improve network-level task performance over time (i.e., actualize learning)? It is widely believed that there is a consistent, synaptic-level learning mechanism in specific brain regions, such as the basal ganglia, that actualizes learning. However, the exact nature of this mechanism remains unclear. Here, we investigate the use of universal synaptic-level algorithms in training connectionist models. Specifically, we propose an algorithm based on reinforcement learning (RL) to generate and apply a simple biologically-inspired synaptic-level learning policy for neural networks. In this algorithm, the action space for each synapse in the network consists of a small increase, decrease, or null action on the connection strength. To test our algorithm, we applied it to a multilayer perceptron (MLP) neural network model. This algorithm yields a static synaptic learning policy that enables the simultaneous training of over 20,000 parameters (i.e., synapses) and consistent learning convergence when applied to simulated decision boundary matching and optical character recognition tasks. The trained networks yield character-recognition performance comparable to identically shaped networks trained with gradient descent. The approach has two significant advantages in comparison to traditional gradient-descent-based optimization methods. First, the robustness of our novel method and its lack of reliance on gradient computations opens the door to new techniques for training difficult-to-differentiate artificial neural networks, such as spiking neural networks (SNNs) and recurrent neural networks (RNNs). Second, the method’s simplicity provides a unique opportunity for further development of local information-driven multiagent connectionist models for machine intelligence analogous to cellular automata.

A Gradient Algorithm for Neural-Network-Based Reinforcement Learning

Gradient Q : A Unified Algorithm with Function Approximation for Reinforcement Learning

Natural Gradient Based Reinforcement Learning Algorithm Using Active Stimulating

Function Gradient Approximation with Random Shallow ReLU Networks with Control Applications

An Experimental Comparison Between Temporal Difference and Residual Gradient with Neural Network Approximation

Multi-Agent Reinforcement Learning Algorithm Based On Neural Networks

Gradient-Free Neural Network Training via Synaptic-Level Reinforcement Learning

Reinforcement Learning for Learning Rate Control.

Gradient Information Matters in Policy Optimization by Back-propagating through Model

Bellman Gradient Iteration for Inverse Reinforcement Learning.

Reinforcement learning for learning rate control

A Novel Reinforcement Learning Control for a Class of Strict-feedback Discrete-time Systems Via Multi-Gradient Recursive.

Plateau Phenomenon in Gradient Descent Training of ReLU networks: Explanation, Quantification and Avoidance

Policy Gradient Reinforcement Learning for Parameterized Continuous-Time Optimal Control

On the Convergence of Discounted Policy Gradient Methods

Deep Reinforcement Learning in Finite-Horizon to Explore the Most Probable Transition Pathway

The Reinforce Policy Gradient Algorithm Revisited

An Automatic Driving Control Method Based on Deep Deterministic Policy Gradient

Actor-critic Algorithm with Incremental Dual Natural Policy Gradient

Interactive Gradient Algorithm For Artificial Neural Networks