Abstract:Conventional artificial neural networks perform functional mappings from their input space to their output space. The synaptic weights encode information about the mapping in a manner analogous to long-term memory in biological systems. This paper presents a method of designing neural networks where recurrent signal loops store this knowledge in a manner analogous to short-term memory. The synaptic weights of these networks encode a learning algorithm. This gives these networks the ability to dynamically learn any functional mapping from a (possibly very large) set, without changing any synaptic weights. These networks are adaptive dynamic systems. Learning is online continually taking place as part of the network's overall behavior instead of a separate, externally driven process. We present four higher order fixed-weight learning networks. Two of these networks have standard backpropagation embedded in their synaptic weights. The other two utilize a more efficient gradient-descent-based learning rule. This new learning scheme was discovered by examining variations in fixed-weight topology. We present empirical tests showing that all these networks were able to successfully learn functions from both discrete (Boolean) and continuous function sets. Largely, the networks were robust with respect to perturbations in the synaptic weights. The exception was the recurrent connections used to store information. These required a tight tolerance of 0.5%. We found that the cost of these networks scaled approximately in proportion to the total number of synapses. We consider evolving fixed weight networks tailored to a specific problem class by analyzing the meta-learning cost surface of the networks presented.

Recurrent function networks model and learning algorithm

Identification of Nonlinear Systems Using Robust Stable Recurrent High Order Neural Networks

Improved delay-dependent stability criteria for time-delayed neural networks

An estimation of the domain of attraction for recurrent neural networks with time-varying delays

New Approach for Robust Stability Analysis of Delayed Recurrent Neural Networks

Improved Delay-Dependent Stability Condition of Discrete Recurrent Neural Networks with Time-Varying Delays

Functional Network Method for the Identification of Nonlinear Systems

Learning Fixed Points of Recurrent Neural Networks by Reparameterizing the Network Model

Discussion of Stability in a Class of Models on Recurrent Radial Basis Function Neural Networks

New results on robust exponential stability for discrete recurrent neural networks with time-varying delays

Continuous Attractors of Discrete-Time Recurrent Neural Networks

Stability analysis of a class of discrete-time recurrent neural networks: An LMI approach

Fixed-weight on-line learning

Qualitative Analysis for Recurrent Neural Networks with Linear Threshold Transfer Functions

Robust Exponential Stability Analysis of a Larger Class of Discrete-Time Recurrent Neural Networks

A SYMMETRIC TERNARY LOGIC BASED MULTI-VALUED LEARNING NETWORK

Dynamics Analysis and Analog Associative Memory of Networks with LT Neurons.

An Augmented Lagrangian Method for Training Recurrent Neural Networks

Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness

Global Exponential Stability of Recurrent Neural Networks with Time-Varying Delay