A Convergent Standard of The On-line Learning Algorithm

ma hongzhao,yang yuling

1994-01-01

Abstract:Abstract It puts forward a convergent standard of the algorithm of the on-line learning of the B-P neural network by the theory and the simulation experiment of the function in this article.

What problem does this paper attempt to address?

Convergence Analysis of General Evolutionary Algorithms

PENG Fuming,YAO Min,BAI Shunke

DOI: https://doi.org/10.3724/sp.j.1087.2013.01571

2013-01-01

Abstract:Traditional Evolutionary Algorithm(EA) convergence research focuses on specific algorithm;consequently the conclusion is only suitable for some specific algorithm.In order to study the convergence of all EAs,this paper presented a general EA including EAs of all operator types.A probability space was set up for the purpose of studying the algorithm's convergence,and all terms on the algorithm were strictly defined in mathematical language,and seven theorems related to the algorithm's convergence were completely proved in the probability space.One of the theorems found the sufficient and necessary conditions for the algorithm's convergence in probability.More importantly,these theorems are suitable to all types of EAs.A system composed of these theorems was established,which could be used to guide the EA design and judge the correctness of an EA theoretically.
Consensus under Bounded Noise in Discrete Network Systems: an Algorithm with Fast Convergence and High Accuracy.

Jianping He,Mengjie Zhou,Peng Cheng,Ling Shi,Jiming Chen

DOI: https://doi.org/10.1109/tcyb.2015.2491368

IF: 11.8

2016-01-01

IEEE Transactions on Cybernetics

Abstract:Most existing works investigate consensus with noise following a certain distribution, e.g., Gaussian distribution, with fixed expectation and variance, which may not be satisfied in practical applications. This paper investigates the discrete system consensus under bounded noise, which is important and practical problem. We first provide necessary and sufficient conditions for the convergence of consensus under bounded noise. To be more general, we derive an analytical bound to show the max-min difference between the nodes' states when the general consensus algorithm converges to a stable state. Then, a novel consensus algorithm, fast consensus under bounded noise (FCBN), is proposed to eliminate the accumulative error caused by the bounded noise. It is proved that FCBN has a faster convergence speed and a higher consensus accuracy than general consensus algorithms. Extensive simulations demonstrate the effectiveness of the proposed algorithm.
A New Convergence Property of Online BP Learning

Rui Zhang,Le Yang,Wei Wang

DOI: https://doi.org/10.1109/icist.2011.5765243

2011-01-01

Abstract:The feedforward neural networks trained with the online backpropagation (BP) learning algorithm have been widely studied in various areas of scientific research and engineering applications. In this paper we further study the convergence property of the online BP learning algorithm. Unlike the existing convergence analysis mainly focusing on the convergence of the gradient sequence of the error functions, we prove a convergence theorem for the sequence of the error functions itself.
Analysis of Boundedness and Convergence of Online Gradient Method for Two-Layer Feedforward Neural Networks

Lu Xu,Jinshu Chen,Defeng Huang,Jianhua Lu,Licai Fang

DOI: https://doi.org/10.1109/tnnls.2013.2257845

IF: 14.255

2013-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:This paper presents a theoretical boundedness and convergence analysis of online gradient method for the training of two-layer feedforward neural networks. The well-known linear difference equation is extended to apply to the general case of linear or nonlinear activation functions. Based on this extended difference equation, we investigate the boundedness and convergence of the parameter sequence of concern, which is trained by finite training samples with a constant learning rate. We show that the uniform upper bound of the parameter sequence, which is very important in the training procedure, is the solution of an inequality regarding the bound. It is further verified that, for the case of linear activation function, a solution always exists and, moreover, the parameter sequence can be uniformly upper bounded, while for the case of nonlinear activation function, some simple adjustment methods on the training set or the activation function can be derived to improve the boundedness property. Then, for the convergence analysis, it is shown that the parameter sequence can converge into a zone around an optimal solution at which the error function attains its global minimum, where the size of the zone is associated with the learning rate. Particularly, for the case of perfect modeling, a strong global convergence result, where the parameter sequence can always converge to an optimal solution, is proved.
The Improved Rapid Convergence Algorithm of the Connecting Rights in the BP Network

Jun Xie,Peng Lin,Huizhen Liang,Minghui Lu

DOI: https://doi.org/10.1109/KAM.2008.124

2008-01-01

Abstract:Based on analyzing fundamental algorithms principle of Back Propagation Network, in view of the limitations of BP network algorithm, this paper proposed the homologous improved algorithms from two respects: accelerate the learning speed of BP network and advance the convergence degree of network. By means of increasing the discrepancy factors of neuron function, the traditional function is optimized and improved, so that the output value break away from local minimal, greatly increasing the convergence speed of network; In the circle iterate expression, by means of increasing the momentum factor, it can filtrate the high frequency deviation of curved face in weight space, so as to widen the space of effective weight, and avoid the oscillation in the network learning process, enormously decrease the time of network learning.
A Multilayer Complex Neural Network Training Algorithm and Its Application in Adaptive Equalization

Li Chunguang,Liao Xiaofeng,Wu Zhongfu,Yu Juebang

DOI: https://doi.org/10.1007/s11767-001-0046-z

2001-01-01

Abstract:In this paper, the layer-by-layer optimizing algorithm for training multilayer neural network is extended for the case of a multilayer neural network whose inputs, weights, and activation functions are all complex. The updating of the weights of each layer in the network is based on the recursive least squares method. The performance of the proposed algorithm is demonstrated with application in adaptive complex communication channel equalization.
Fast and Strong Convergence of Online Learning Algorithms.

Zheng-Chu Guo,Lei Shi

DOI: https://doi.org/10.1007/s10444-019-09707-8

2019-01-01

Advances in Computational Mathematics

Abstract:In this paper, we study the online learning algorithm without explicit regularization terms. This algorithm is essentially a stochastic gradient descent scheme in a reproducing kernel Hilbert space (RKHS). The polynomially decaying step size in each iteration can play a role of regularization to ensure the generalization ability of online learning algorithm. We develop a novel capacity dependent analysis on the performance of the last iterate of online learning algorithm. This answers an open problem in learning theory. The contribution of this paper is twofold. First, our novel capacity dependent analysis can lead to sharp convergence rate in the standard mean square distance which improves the results in the literature. Second, we establish, for the first time, the strong convergence of the last iterate with polynomially decaying step sizes in the RKHS norm. We demonstrate that the theoretical analysis established in this paper fully exploits the fine structure of the underlying RKHS, and thus can lead to sharp error estimates of online learning algorithm.
New and Rapidly Converging BP Neural Network Algorithm

王赟松,褚福磊,何永勇,郭丹

DOI: https://doi.org/10.3969/j.issn.1000-1298.2004.06.050

2004-01-01

Abstract:The feature of a slow converging of a standard BP neural network algorithm is the main cause of its limitation for a wide application. A new BP algorithm, Levenberg-Marquardt back-propagation algorithm, which could converge rapidly, was developed in this paper based on the least square method and the standard BP algorithm. The experimental result proved that LMBP converged very fast and had a good stability compared with the standard BP algorithm, which meat that LMBP algorithm was suitable for a big network with hundreds of weight values or high training precision demand. The superiority of the algorithm over others is notable if the computer memory is large enough.
Study on Fast Intelligent Learning Algorithm of Neural Network(i)

Yang Wei

2001-01-01

Abstract:One rapid computation algorithm is studied,simultantly,the global convergence is considered.Improve the construction of BP network by genetic algorithm;adjust the probabilities of crossover and mutation by the fuzzy control method.Then,disscuss the effective methods to improve the BP network in practice.
A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks

Asaf Noy,Yi Xu,Yonathan Aflalo,Lihi Zelnik-Manor,Rong Jin

DOI: https://doi.org/10.48550/arXiv.2101.04243

IF: 5.414

2021-01-12

Machine Learning

Abstract:Deep neural networks' remarkable ability to correctly fit training data when optimized by gradient-based algorithms is yet to be fully understood. Recent theoretical results explain the convergence for ReLU networks that are wider than those used in practice by orders of magnitude. In this work, we take a step towards closing the gap between theory and practice by significantly improving the known theoretical bounds on both the network width and the convergence time. We show that convergence to a global minimum is guaranteed for networks with widths quadratic in the sample size and linear in their depth at a time logarithmic in both. Our analysis and convergence bounds are derived via the construction of a surrogate network with fixed activation patterns that can be transformed at any time to an equivalent ReLU network of a reasonable size. This construction can be viewed as a novel technique to accelerate training, while its tight finite-width equivalence to Neural Tangent Kernel (NTK) suggests it can be utilized to study generalization as well.
A Comparison and Improvement of Online Learning Algorithms for Sequence Labeling.

Zhengyan He,Houfeng Wang

2012-01-01

Abstract:Sequence labeling models like conditional random fields have been successfully applied in a variety of NLP tasks. However, as the size of label set and dataset grows, the learning speed of batch algorithms like L-BFGS quickly becomes computationally unacceptable. Several online learning methods have been proposed in large scale setting, yet little effort has been made to compare the performance of these algorithms. Comparison is often carried out on a few datasets with fine tuned parameters for specific algorithm. In this paper, we investigate and compare several online learning algorithms for sequence labeling with datasets varying in scale, feature design and label set. We find that Dual Coordinate Ascent (DCA) is robust across datasets even without careful tuning of parameter. Furthermore, a recently proposed variant of Stochastic Gradient Descent (SGD), Adaptive online gradient Descent based on feature Frequency information (ADF), has very fast training speed compared with plain SGD, but fails to converge under certain conditions. Finally, We propose a simple modification of ADF, which bears comparable convergence speed with ADF, and is consistently better than plain SGD. TITLE AND ABSTRACT IN CHINESE
Convergence Analysis of Two-layer Neural Networks with ReLU Activation

Yuanzhi Li,Yang Yuan

DOI: https://doi.org/10.48550/arxiv.1705.09886

2017-01-01

Abstract:In recent years, stochastic gradient descent (SGD) based techniques has become the standard tools for training neural networks. However, formal theoretical understanding of why SGD can train neural networks in practice is largely missing. In this paper, we make progress on understanding this mystery by providing a convergence analysis for SGD on a rich subset of two-layer feedforward networks with ReLU activations. This subset is characterized by a special structure called "identity mapping". We prove that, if input follows from Gaussian distribution, with standard $O(1/\sqrt{d})$ initialization of the weights, SGD converges to the global minimum in polynomial number of steps. Unlike normal vanilla networks, the "identity mapping" makes our network asymmetric and thus the global minimum is unique. To complement our theory, we are also able to show experimentally that multi-layer networks with this mapping have better performance compared with normal vanilla networks. Our convergence theorem differs from traditional non-convex optimization techniques. We show that SGD converges to optimal in "two phases": In phase I, the gradient points to the wrong direction, however, a potential function $g$ gradually decreases. Then in phase II, SGD enters a nice one point convex region and converges. We also show that the identity mapping is necessary for convergence, as it moves the initial point to a better place for optimization. Experiment verifies our claims.
On the Convergence of Bounded Agents

David Abel,André Barreto,Hado van Hasselt,Benjamin Van Roy,Doina Precup,Satinder Singh

2023-07-21

Abstract:When has an agent converged? Standard models of the reinforcement learning problem give rise to a straightforward definition of convergence: An agent converges when its behavior or performance in each environment state stops changing. However, as we shift the focus of our learning problem from the environment's state to the agent's state, the concept of an agent's convergence becomes significantly less clear. In this paper, we propose two complementary accounts of agent convergence in a framing of the reinforcement learning problem that centers around bounded agents. The first view says that a bounded agent has converged when the minimal number of states needed to describe the agent's future behavior cannot decrease. The second view says that a bounded agent has converged just when the agent's performance only changes if the agent's internal state changes. We establish basic properties of these two definitions, show that they accommodate typical views of convergence in standard settings, and prove several facts about their nature and relationship. We take these perspectives, definitions, and analysis to bring clarity to a central idea of the field.

Machine Learning,Artificial Intelligence
On Convergence of Training Loss Without Reaching Stationary Points

Jingzhao Zhang,Haochuan Li,Suvrit Sra,Ali Jadbabaie

2021-01-01

Abstract: It is a well-known fact that nonconvex optimization is computationally intractable in the worst case. As a result, theoretical analysis of optimization algorithms such as gradient descent often focuses on local convergence to stationary points where the gradient norm is zero or negligible. In this work, we examine the disconnect between the existing theoretical analysis of gradient-based algorithms and actual practice. Specifically, we provide numerical evidence that in large-scale neural network training, such as in ImageNet, ResNet, and WT103 + TransformerXL models, the Neural Network weight variables do not converge to stationary points where the gradient of the loss function vanishes. Remarkably, however, we observe that while weights do not converge to stationary points, the value of the loss function converges. Inspired by this observation, we propose a new perspective based on ergodic theory of dynamical systems. We prove convergence of the distribution of weight values to an approximate invariant measure (without smoothness assumptions) that explains this phenomenon. We further discuss how this perspective can better align the theory with empirical observations.
Global Convergence of Delayed Neural Network Systems.

Wenlian Lu,Tianping Chen

DOI: https://doi.org/10.1142/s0129065703001534

IF: 6.325

2003-01-01

International Journal of Neural Systems

Abstract:In this paper, without assuming the boundedness, strict monotonicity and differentiability of the activation functions, we utilize a new Lyapunov function to analyze the global convergence of a class of neural networks models with time delays. A new sufficient condition guaranteeing the existence, uniqueness and global exponential stability of the equilibrium point is derived. This stability criterion imposes constraints on the feedback matrices independently of the delay parameters. The result is compared with some previous works. Furthermore, the condition may be less restrictive in the case that the activation functions are hyperbolic tangent.
The Research on the Relation of Self-Learning Ratio and the Convergence Speed in BP Networks

WN WEN,SX LIU,ZY ZHOU

DOI: https://doi.org/10.1109/imtc.1994.352107

1994-01-01

Abstract:The relation of self-learning ratio and the convergence speed in BP network is proposed in this paper. In theory, only when the self-learning ratio /spl muspl rarr/0, the real gradient descent can be got, and the computation will converge to a certain local minimum point. But, a too small /spl mu/ will cause a slow convergence speed and a too large /spl mu/ may cause divergence. On the base of mathematical analysis and some computer simulations, the relation formula is given out as follows: n=ln[/spl epsi|W(0)-W*|]/ln(1-/spl mu/a) where n is the amount of iterative, /spl mu/ is self-learning ratio, w(0) is the original weight and w* is the best weight, /spl epsi/ is the precision requirement, a is the slope of gradient imitative straight line. It is also proposed for a method to determine a better self-learning ratio.<>
A Novel Genetic Neural Network Algorithm with Link Switches and Its Application in University Professional Course Evaluation

Honghai Ji,Jinyao Zhou,Shida Liu,Li Wang,Lingling Fan

DOI: https://doi.org/10.1155/2022/9564443

IF: 3.12

2022-05-25

Computational Intelligence and Neuroscience

Abstract:This study exploits a novel enhanced genetic neural network algorithm with link switches (EGA-NNLS) to model the professional university course evaluating system. Various indices should be employed to evaluate the learning effect of a professional course comprehensively and objectively, and the traditional artificial evaluation methods cannot achieve this goal. The presented data-driven modeling method, EGA-NNLS, combines a neural network with link switches (NN-LS) with an enhanced genetic algorithm (EGA) and the Levenberg–Marquardt (LM) algorithm. It employs an optimized network structure combined with EGA and NN-LS to learn the relationships between the system's input and output from historical data and uses the network's gradient information via the LM algorithm. Compared with the traditional backpropagation neural network (BPNN), EGA-NNLS achieves a faster convergence speed and higher evaluation precision. In order to verify the efficiency of EGA-NNLS, it is applied to a collection of experimental data for modeling the professional university course evaluating system.

mathematical & computational biology,neurosciences
On The Global Convergence Of Online RLHF With Neural Parametrization

Mudit Gaur,Amrit Singh Bedi,Raghu Pasupathy,Vaneet Aggarwal

2024-10-21

Abstract:The importance of Reinforcement Learning from Human Feedback (RLHF) in aligning large language models (LLMs) with human values cannot be overstated. RLHF is a three-stage process that includes supervised fine-tuning (SFT), reward learning, and policy learning. Although there are several offline and online approaches to aligning LLMs, they often suffer from distribution shift issues. These issues arise from the inability to accurately capture the distributional interdependence between the reward learning and policy learning stages. Consequently, this has led to various approximated approaches, but the theoretical insights and motivations remain largely limited to tabular settings, which do not hold in practice. This gap between theoretical insights and practical implementations is critical. It is challenging to address this gap as it requires analyzing the performance of AI alignment algorithms in neural network-parameterized settings. Although bi-level formulations have shown promise in addressing distribution shift issues, they suffer from the hyper-gradient problem, and current approaches lack efficient algorithms to solve this. In this work, we tackle these challenges employing the bi-level formulation laid out in Kwon et al. (2024) along with the assumption \emph{Weak Gradient Domination} to demonstrate convergence in an RLHF setup, obtaining a sample complexity of $\epsilon^{-\frac{7}{2}}$ . Our key contributions are twofold: (i) We propose a bi-level formulation for AI alignment in parameterized settings and introduce a first-order approach to solve this problem. (ii) We analyze the theoretical convergence rates of the proposed algorithm and derive state-of-the-art bounds. To the best of our knowledge, this is the first work to establish convergence rate bounds and global optimality for the RLHF framework in neural network-parameterized settings.

Machine Learning
An Optimal Algorithm for Online Non-Convex Learning

Lin Yang,Lei Deng,Mohammad H. Hajiesmaili,Cheng Tan,Wing Shing Wong

DOI: https://doi.org/10.1145/3224420

2018-01-01

Proceedings of the ACM on Measurement and Analysis of Computing Systems

Abstract:In many online learning paradigms, convexity plays a central role in the derivation and analysis of online learning algorithms. The results, however, fail to be extended to the non-convex settings, which are necessitated by tons of recent applications. The Online Non-Convex Learning problem generalizes the classic Online Convex Optimization framework by relaxing the convexity assumption on the cost function (to a Lipschitz continuous function) and the decision set. The state-of-the-art result for ønco demonstrates that the classic Hedge algorithm attains a sublinear regret of O(√T log T). The regret lower bound for øco, however, is Omega(√T), and to the best of our knowledge, there is no result in the context of the ønco problem achieving the same bound. This paper proposes the Online Recursive Weighting algorithm with regret of O(√T), matching the tight regret lower bound for the øco problem, and fills the regret gap between the state-of-the-art results in the online convex and non-convex optimization problems.
A Recipe for Global Convergence Guarantee in Deep Neural Networks

Kenji Kawaguchi,Qingyun Sun

DOI: https://doi.org/10.1609/aaai.v35i9.16984

2021-05-18

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:Existing global convergence guarantees of (stochastic) gradient descent do not apply to practical deep networks in the practical regime of deep learning beyond the neural tangent kernel (NTK) regime. This paper proposes an algorithm, which is ensured to have global convergence guarantees in the practical regime beyond the NTK regime, under a verifiable condition called the expressivity condition. The expressivity condition is defined to be both data-dependent and architecture-dependent, which is the key property that makes our results applicable for practical settings beyond the NTK regime. On the one hand, the expressivity condition is theoretically proven to hold data-independently for fully-connected deep neural networks with narrow hidden layers and a single wide layer. On the other hand, the expressivity condition is numerically shown to hold data-dependently for deep (convolutional) ResNet with batch normalization with various standard image datasets. We also show that the the proposed algorithm has generalization performances comparable with those of the heuristic algorithm, with the same hyper-parameters and total number of iterations. Therefore, the proposed algorithm can be viewed as a step towards providing theoretical guarantees for deep learning in the practical regime.

A Convergent Standard of The On-line Learning Algorithm

Convergence Analysis of General Evolutionary Algorithms

Consensus under Bounded Noise in Discrete Network Systems: an Algorithm with Fast Convergence and High Accuracy.

A New Convergence Property of Online BP Learning

Analysis of Boundedness and Convergence of Online Gradient Method for Two-Layer Feedforward Neural Networks

The Improved Rapid Convergence Algorithm of the Connecting Rights in the BP Network

A Multilayer Complex Neural Network Training Algorithm and Its Application in Adaptive Equalization

Fast and Strong Convergence of Online Learning Algorithms.

New and Rapidly Converging BP Neural Network Algorithm

Study on Fast Intelligent Learning Algorithm of Neural Network(i)

A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks

A Comparison and Improvement of Online Learning Algorithms for Sequence Labeling.

Convergence Analysis of Two-layer Neural Networks with ReLU Activation

On the Convergence of Bounded Agents

On Convergence of Training Loss Without Reaching Stationary Points

Global Convergence of Delayed Neural Network Systems.

The Research on the Relation of Self-Learning Ratio and the Convergence Speed in BP Networks

A Novel Genetic Neural Network Algorithm with Link Switches and Its Application in University Professional Course Evaluation

On The Global Convergence Of Online RLHF With Neural Parametrization

An Optimal Algorithm for Online Non-Convex Learning

A Recipe for Global Convergence Guarantee in Deep Neural Networks