Abstract:Neural Networks (NN) have become important tools that have demonstrated their value solving complex problems regarding pattern recognition, natural language processing, automatic speech recognition, among others. Recently, the number of applications that require running the training process at the front-end in an online manner have increased dramatically. Unfortunately, in state-of-the-art (SoA) methods, this training process is an unbounded function of the initial conditions. Thus, there is no insight on the number of epochs required, making the online training a difficult problem. Speeding up the training process plays a key role in machine learning. In this work, an algorithm for dynamic learning rate is proposed based on recent results from fixed-time stability of continuous-time nonlinear systems, which ensures a convergence time bound to the equilibrium point independently of the initial conditions. We show experimentally that our discrete-time implementation presents promising results, proving that the number of epochs required for the training remains bounded, independently of the initial weights. This constitutes an important feature toward learning systems with real-time constraints. The efficiency of the method proposed is illustrated under different scenarios, including the public database MNIST, which shows that out algorithm outperforms SoA methods in terms of the number of epoch required for the training.

Adaptive Learning with Guaranteed Stability for Discrete-Time Recurrent Neural Networks

Improved Delay-Dependent Stability Condition of Discrete Recurrent Neural Networks with Time-Varying Delays

Adaptive Smoothing Gradient Learning for Spiking Neural Networks.

Improved delay-dependent stability criteria for time-delayed neural networks

Dynamic Learning Rate for Neural Networks: A Fixed-Time Stability Approach

Improved Delay-Dependent Stability Results of Recurrent Neural Networks

A Study of Asymptotic Stability for Delayed Recurrent Neural Networks.

New Stability Results of Discrete-Time Recurrent Neural Networks with Time-Varying Delays

Improved Exponential Stability Criteria for Recurrent Neural Networks with Time-Varying Discrete and Distributed Delays

Delay-Dependent Stability Of Recurrent Neural Networks With Time-Varying Delay

Exponential Stability of Recurrent Neural Networks with Time-Varying Discrete and Distributed Delays

Recurrent function networks model and learning algorithm

Study of applied learning rates of recurrent neural networks

New Delay-Dependent Stability Results for Discrete-Time Recurrent Neural Networks with Time-Varying Delay

New Criteria for Exponential Stability of Delayed Recurrent Neural Networks.

Stability Analysis for Recurrent Neural Networks with Time-Varying Delay

Impulsive Stabilization of Periodic Solutions of Recurrent Neural Networks with Discrete and Distributed Delays

Simplified Exponential Stability Analysis for Recurrent Neural Networks with Discrete and Distributed Time-Varying Delays

Delay Dependent Stability Conditions of Static Recurrent Neural Networks: a Non-Linear Convex Combination Method

Delay-dependent stability for static recurrent neural networks via a piecewise delay approach

Delay-interval-dependent Stability of Recurrent Neural Networks with Time-Varying Delay