Study on Fast Intelligent Learning Algorithm of Neural Network(i)

Yang Wei

2001-01-01

Abstract:One rapid computation algorithm is studied,simultantly,the global convergence is considered.Improve the construction of BP network by genetic algorithm;adjust the probabilities of crossover and mutation by the fuzzy control method.Then,disscuss the effective methods to improve the BP network in practice.

What problem does this paper attempt to address?

The Study on Neural Network Intelligent Method Based on Genetic Algorithm

Hongtao Wang

DOI: https://doi.org/10.4028/www.scientific.net/amr.271-273.546

2011-01-01

Advanced Materials Research

Abstract:The paper gives the hybrid computational intelligence learning algorithm with global convergence, which is combined by BP algorithm and genetic algorithm. This algorithm connects the strengths of the BP algorithm and genetic algorithms. It not only has faster convergence, but also has a good global convergence property. The computer simulation results show that the hybrid algorithm is significantly better than the genetic algorithm and BP algorithm.
Fast Algorithm for Adaptive Generalized Predictive Control Based on Bp Neural Networks

QA Li,SQ Wang

DOI: https://doi.org/10.1109/icmlc.2004.1382282

2004-01-01

Abstract:The traditional way of reducing the on-line computing time required for the standard adaptive generalized predictive control (GPC), is by using a short predictive horizon or a short control horizon. However, it breaches the intrinsic principle of this long range receding horizon strategy, and sometimes it could lead to poor control performance in some processes. A kind of fast algorithm for self-tuning GPC (FGPC) is presented by using back propagation (BP) neural networks. The algorithm involves the derivation of the GPC in a new way and the training of BP neural networks to represent nonlinear relations between the GPC controller coefficients and system open-loop parameters, control horizon, smoothing factor and control weighting factor. The method developed does not involve the Diophantine equation and avoids the matrix operation, therefore substantially alleviates the mathematical computation problems associated with the standard adaptive GPC, especially for a long control horizon. The efficacy of this algorithm is demonstrated with a comparative simulation study.
Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and Bp Algorithms

Yan-Peng Liu,Ming-Guang Wu,Ji-Xin Qian

DOI: https://doi.org/10.1007/11759966_105

2006-01-01

Abstract:Ant colony optimization (ACO) algorithm has the powerful ability of searching the global optimal solution, and backpropagation (BP) algorithm has the feature of rapid convergence on the local optima. The proper hybrid of the two algorithms (ACO-BP) may accelerate the evolving speed of neural networks and improve the forecasting precision of the well-trained networks. ACO-BP scheme adopts ACO to search the optimal combination of weights in the solution space, and then uses BP algorithm to obtain the accurate optimal solution quickly. The ACO-BP and BP algorithms were applied to the problems of function approaching and modeling quantitative structure-activity relationships of Herbicides. Experiment results show that the proposed ACO-BP scheme is more efficient and effective than BP algorithm. Furthermore, ACO-BP reliably performs well when the number of hidden nodes varies.
Optimization of Backpropagation Neural Network under the Adaptive Genetic Algorithm

Junxi Zhang,Shiru Qu

DOI: https://doi.org/10.1155/2021/1718234

IF: 2.3

2021-01-01

Complexity

Abstract:This study is to explore the optimization of the adaptive genetic algorithm (AGA) in the backpropagation (BP) neural network (BPNN), so as to expand the application of the BPNN model in nonlinear issues. Traffic flow prediction is undertaken as a research case to analyse the performance of the optimized BPNN. Firstly, the advantages and disadvantages of the BPNN and genetic algorithm (GA) are analyzed based on their working principles, and the AGA is improved and optimized. Secondly, the optimized AGA is applied to optimize the standard BPNN, and the optimized algorithm is named as OAGA-BPNN. Finally, three different cases are proposed based on the actual scenario of traffic flow prediction to analyse the optimized algorithm on the matrix laboratory (MATLAB) platform by simulation. The results show that the average error distribution of the GA-BPNN algorithm is about 1% with small fluctuation range, better calculation accuracy, and generalization performance in contrast to the BPNN. The average output error of the AGA-BPNN fluctuates around 0 and remains in a relatively stable range as a whole in contrast to that of GA-BPNN; the maximum fitness level keeps increasing during the evolution process but approaches the average value in later process, so the population diversity is hard to be guaranteed. The output error of the OAGA-BPNN fluctuates little compared with that of AGA-BPNN, and its maximum fitness continues to increase in the evolution process with guaranteed population diversity. In short, the OAGA-BPNN algorithm can achieve the best performance in terms of calculation accuracy, generalization performance, and population evolution.
An optimizing BP neural network algorithm based on genetic algorithm

Shifei Ding,Chunyang Su,Junzhao Yu

DOI: https://doi.org/10.1007/s10462-011-9208-z

IF: 9.588

2011-02-18

Artificial Intelligence Review

Abstract:A back-propagation (BP) neural network has good self-learning, self-adapting and generalization ability, but it may easily get stuck in a local minimum, and has a poor rate of convergence. Therefore, a method to optimize a BP algorithm based on a genetic algorithm (GA) is proposed to speed the training of BP, and to overcome BP’s disadvantage of being easily stuck in a local minimum. The UCI data set is used here for experimental analysis and the experimental result shows that, compared with the BP algorithm and a method that only uses GA to learn the connection weights, our method that combines GA and BP to train the neural network works better; is less easily stuck in a local minimum; the trained network has a better generalization ability; and it has a good stabilization performance.

computer science, artificial intelligence
Self-adaptation Optimize BP Neural Network Design Based on the Genetic Algorithms

CHAI Yi,YIN Hong-peng,LI Da-jie,ZHANG Ke

DOI: https://doi.org/10.3969/j.issn.1000-582x.2007.04.021

2007-01-01

Abstract:BP(Back Propagation) Neural networks is in the presence of the local optimization in the Neural networks training.The algorithm have slow convergence and the local convergence problem which impact the neural networks work performance.In order to cover these shortcomings and solves the size's hugeness and the low efficiency of the net problem in the traditional NN designing,the action principles of BP-Neural network's structure are analyzed,and a new method is formed which is confirmed from the Enhance genetic algorithms(EGA).The method can identify network configuration and network training methods.By adopting the number coding,self-adaptable multi-point variations operation,this method can effectively reduce the network size and the network convergence time,increase the network training speed.Tomatoes disease diagnosis examples illustrate the feasibility of this approach.
Comparative Study on Fast Learning Algorithms of BP Networks

高雪鹏,丛爽

DOI: https://doi.org/10.3321/j.issn:1001-0920.2001.02.009

2001-01-01

Abstract:A comparative study on some typical faster learning algorithms ofBP networks is proposed. Firstly, the advantages and disadvantages of two main categories, the fast algorithms based on standard steepest descent and standard numerical optimization, are analysed by means of their work principles. Then the comparative study on convergence times and error performance of training of those faster algorithms is done by two numerical examples. The study can be used for the reference in selecting learning algorithms and developing high performance algorithms.
New and Rapidly Converging BP Neural Network Algorithm

王赟松,褚福磊,何永勇,郭丹

DOI: https://doi.org/10.3969/j.issn.1000-1298.2004.06.050

2004-01-01

Abstract:The feature of a slow converging of a standard BP neural network algorithm is the main cause of its limitation for a wide application. A new BP algorithm, Levenberg-Marquardt back-propagation algorithm, which could converge rapidly, was developed in this paper based on the least square method and the standard BP algorithm. The experimental result proved that LMBP converged very fast and had a good stability compared with the standard BP algorithm, which meat that LMBP algorithm was suitable for a big network with hundreds of weight values or high training precision demand. The superiority of the algorithm over others is notable if the computer memory is large enough.
The Improved Rapid Convergence Algorithm of the Connecting Rights in the BP Network

Jun Xie,Peng Lin,Huizhen Liang,Minghui Lu

DOI: https://doi.org/10.1109/KAM.2008.124

2008-01-01

Abstract:Based on analyzing fundamental algorithms principle of Back Propagation Network, in view of the limitations of BP network algorithm, this paper proposed the homologous improved algorithms from two respects: accelerate the learning speed of BP network and advance the convergence degree of network. By means of increasing the discrepancy factors of neuron function, the traditional function is optimized and improved, so that the output value break away from local minimal, greatly increasing the convergence speed of network; In the circle iterate expression, by means of increasing the momentum factor, it can filtrate the high frequency deviation of curved face in weight space, so as to widen the space of effective weight, and avoid the oscillation in the network learning process, enormously decrease the time of network learning.
Research on Improvement of Convergence Speed of LMBP Algorithm in Neural Network

Li Jiongcheng,Huang Hanxiong

DOI: https://doi.org/10.3321/j.issn:1002-8331.2006.16.016

2006-01-01

Abstract:The reason is analyzed why the convergence of standard BP algorithm is slow.Detail research is carried out on Levenberg-Marquardt BP(LMBP),which is the fastest improved version of BP.It is found that the calculation of inverse matrix of [JTJ+μkI] involved causes primarily the problem.LU decomposition is employed to avoid the time-consuming calculation of the inverse matrix,so as to decrease largely the calculation amount of LMBP.In addition,the steps for the calculation of the augmented Marquardt sensitivity matrix are simplified,which also decreases the calculation amount to a certain extent.The improved LMBP algorithm is implemented by authors with Microsoft Visual C++ 6.It is found that above two improvements caused LMBP to converge rapidly.Many trials are conducted on Traingdx from Matlab,which is based on the steepest descent algorithm,Trainlm from Matlab,which is improved on LMBP in Matlab,LMBP,and ILMBP,which is improved on LMBP by authors.The results show that the mean convergence speed of ILMBP is 23 times quicker than that of LMBP,and 9 times than that of Trainlm.
Study and Application of an Adaptive Genetic BP Neural Network Model

WEN Quan-che,PENG Hong,LI Qiong

DOI: https://doi.org/10.3969/j.issn.1006-9348.2006.12.043

2006-01-01

Abstract:Based on the characteristics of the genetic algorithm and BP nerve network,the paper proposes an adaptive genetic BP neural network model.This model uses an adaptive genetic algorithms to optimize the BP network initial weight first,then carries out the BP network training process.Finally using this model to carry out the triple jump score prediction.The experimental result indicates that this method surpasses traditional BP algorithm,can improve the convergency and the learning capability of the network,and may make the triple jump score prediction more accurate.
Fault Diagnosis Method Based on Improved Genetic Algorithm and Neural Network

Dawei Zhang,Weilin Li,Xiaohua Wu,Xiaofeng Lv

DOI: https://doi.org/10.1109/cieec.2018.8745825

2018-01-01

Abstract:In order to overcome the shortcomings such as slow convergence rate and prone to sink into small locality in BP neural network, adaptive genetic algorithm and BP algorithm are combined to take shape a hybrid algorithm to train artificial neural network. In a specific implementation, firstly, an adaptive genetic algorithm is used to perform multi-point genetic optimization on the initial weight space of the neural network, and better search space is located in the solution space. On this basis, local exact search is performed using BP algorithm, ultimately the global optimum is achieved. This algorithm is simulated based on the fault diagnosis of one certain helicopter's airborne electrical control box and one certain flight control box of aircraft autopilot. The simulation conclusions indicate that the algorithm has faster convergence rate and higher diagnostic accuracy.
An Effective Backpropagation Algorithm for Optimizing BP Neural Network Based on Rough Set and Modified Genetic Algorithm

li wei,he pengju,yang heng,chen ming

DOI: https://doi.org/10.3969/j.issn.1000-2758.2012.04.022

2012-01-01

Xibei Gongye Daxue Xuebao/Journal of Northwestern Polytechnical University

Abstract:Considering that the BP neural network became complex due to the increase of the sample dimension and it fell easily into local maximums or minimums,we combined genetic algorithm and rough set to optimize the BP neural network.Sections 1 through 3 explain our backpropagation algorithm mentioned in the title,which we believe is effective and whose core consists of:(1) rough set was applied to simplify the network by reducing the attribute dimension;(2) modified genetic algorithm was used to globally search the weights and bios and,further,the BP algorithm was to locally optimize them to avoid the network falling into the local extremes.Simulation results,presented in Fig.1 and Table 2 in subsection 3.4,and their analysis indicated preliminarily that prediction accuracy was increased greatly over that of the traditional BP neural network and that generalization was enhanced,thus showing that our backpropagation algorithm is indeed effective.
A Method of Image Classification with Optimized BP Neural Network by Genetic Algorithm

Shen Qian,Liu Chan-juan,Zou Hai-lin,Zhou Shu-sen,Chen Tong-tong

DOI: https://doi.org/10.1109/incos.2015.14

2015-01-01

Abstract:BP neural network is an important and efficient method in machine learning. But there are some drawbacks lying in its local minimum and slow convergence speed. To solve these problems and enhance the performance of BP network, an optimized BP neural network by genetic algorithm is proposed in this paper. Firstly, we design a fitness function based on genetic algorithm for the view of obtaining the best individual, and then optimize the weights and thresholds of the connections between the layers of BP network. The experiments and analysis indicate that the optimized method not only improves the classification percentage, but also obtain satisfactory mean square error.
Optimization Study of BP Neural Network Based on Genetic Algorithm

Zhuye Zhou,Yongheng Cai,Zehui Li

DOI: https://doi.org/10.1109/ICEACE60673.2023.10442575

2023-12-29

Abstract:In today's era of big data, neural networks play a pivotal role in fields such as image recognition and natural language processing. However, traditional Backpropagation (BP) neural networks often encounter challenges when dealing with complex problems, including slow convergence and getting stuck in local optima. To address these issues, this study employs a genetic algorithm as an optimization technique to optimize the weights and thresholds of BP neural networks. Training is conducted on a set of highly nonlinear mapping datasets to obtain the optimal weight and threshold configurations for the GA-BP model. Experimental results demonstrate that compared to traditional BP neural networks, the GA-BP model exhibits a 19.8% improvement in prediction accuracy, an approximately 9.20% increase in the coefficient of determination (R2), a reduction of approximately 62.50% in mean squared error (MSE), and a decrease of about 38.76% in root mean squared error (RMSE). Furthermore, the GA-BP model displays significantly accelerated convergence. The contribution of this research lies in proposing an optimization method that combines genetic algorithms with BP neural networks. This model not only enhances prediction accuracy compared to the standalone BP neural network but also significantly strengthens its generalization capabilities. As a result, it demonstrates substantial practical application potential across diverse datasets and application scenarios.

Computer Science
Genetic algorithms and artificial neural network. (1) Training artificial neural network with extended genetic algorithms

Fangze Chen,Bingzhen Chen,XiaoRong He

1996-01-01

Abstract:IN order to overcome a local optimal solution which often occurs in the training process of artificial neural networks, a new algorithm called EGA is proposed. An adaptive multipoint mutation operator, which can effectively extinguish the swing problem in convergent process, is proposed in this algorithm. So the speed of BP training is improved. A reasonable mutation factor range is found. As a result, the speed and accuracy of training artificial neural networks are much improved.
Study on the fast learning algorithm of single-layer neural networks

ZhaoZhi Feng,Zailu Huang,Shuzi Yang

1997-01-01

Abstract:This paper proposes a new generalized criterion for the training of single-layer neural networks, which leads to a novel fast learning algorithm for single-layer neural network. In order to analyze the convergent properties of this algorithm, a new generalized system identification model is also presented. Experiment results show that this algorithm proposed performs the training of neural network faster than the corresponding learning algorithm given by Karayiannis.
Research and Application of an Optimized BP Neural Network Based on Adaptive Genetic Algorithm

ZHUANG Jia-jun,LIU Qiong

DOI: https://doi.org/10.13190/jbupt.201205.41.252

2012-01-01

Abstract:The population diversity of conventional genetic algorithm can be easily destroyed,which further leads to premature convergence.To solve this problem,based on adaptive genetic algorithm(AGA) proposed by Srinivas,a modified adaptive genetic algorithm(MAGA) is presented by introducing a parameter measuring the population diversity.In this way,the probabilities of crossover and mutation are adjusted automatically according to both population diversity and the trends of fitness values.Since MAGA and back-propagation(BP) algorithm are good at searching global and local optimum respectively,an optimized BP neural network based on MAGA(MAGA+BP) is then presented for traffic classification.The Internet traffic dataset provided by university of Cambridge is introduced for experimental validation.Results show that: MAGA shows better performance on maintaining population diversity,overcomes the premature convergence of AGA and improves the fitness value of resulting optimum by 10.17%;MAGA+BP shows a better performance on Internet traffic classification.
Two-Times Training Algorithm of Neural Network Based on GA

DENG Zheng-hong,HU Qin,ZHENG Yu-shan

DOI: https://doi.org/10.3969/j.issn.1000-7180.2005.03.067

2005-01-01

Abstract:In order to overcome the lack of BP algorithm, Utilizing the advantage of Neural Network and genetic algorithm, a kind of two-times training method based on genetic algorithm is proposed. Genetic algorithm is applied in the weight training of neural network ,and using the two-times training method of the Neural Network come to the conclusion. The efficient method this paper proposed can reducing calculating time.
Studies on Optimization Algorithms for Some Artificial Neural Networks Based on Genetic Algorithm (GA)

Shifei Ding,Xinzheng Xu,Hong Zhu,Jian Wang,Fengxiang Jin

DOI: https://doi.org/10.4304/jcp.6.5.939-946

2011-01-01

Journal of Computers

Abstract:Artificial Neural Networks (ANNs) are the nonlinear and adaptive information processing systems which are combined by numerous processing units, with the characteristics of self-adapting, self-organizing and real- time learning, and play an important in pattern recognition, machine learning and data mining. But we've encountered many problems, such as the selection of the structure and the parameters of the networks, the selection of the learning samples, the selection of the initial values, the convergence of the learning algorithms and so on. Genetic algorithms (GA) is a kind of random search algorithm, on one hand, it simulates the nature selection and evolution, on the other, it has the advantages of good global search abilities and learning the approximate optimal solution without the gradient information of the error functions. In this paper, some optimization algorithms for ANNs with GA are studied. Firstly, an optimizing BP neural network is set up. It is using GA to optimize the connection weights of the neural network, and using GA to optimize both the connection weights and the architecture. Secondly, an optimizing RBF neural network is proposed. It used hybrid encoding method, that is, to encode the network by binary encoding and the weights by real encoding, the network architecture is self-adapted adjusted, the weights are learned, and the network is further adjusted by pseudo- inverse method or LMS method. Then they are used in real world classification tasks, and compared with the modified BP algorithm with adaptive learning rate. Experiments prove that the network got by this method has a better architecture and stronger classification ability, and the time of constructing the network artificially is saved. The algorithm is a self-adapted and intelligent learning algorithm.

Study on Fast Intelligent Learning Algorithm of Neural Network(i)

The Study on Neural Network Intelligent Method Based on Genetic Algorithm

Fast Algorithm for Adaptive Generalized Predictive Control Based on Bp Neural Networks

Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and Bp Algorithms

Optimization of Backpropagation Neural Network under the Adaptive Genetic Algorithm

An optimizing BP neural network algorithm based on genetic algorithm

Self-adaptation Optimize BP Neural Network Design Based on the Genetic Algorithms

Comparative Study on Fast Learning Algorithms of BP Networks

New and Rapidly Converging BP Neural Network Algorithm

The Improved Rapid Convergence Algorithm of the Connecting Rights in the BP Network

Research on Improvement of Convergence Speed of LMBP Algorithm in Neural Network

Study and Application of an Adaptive Genetic BP Neural Network Model

Fault Diagnosis Method Based on Improved Genetic Algorithm and Neural Network

An Effective Backpropagation Algorithm for Optimizing BP Neural Network Based on Rough Set and Modified Genetic Algorithm

A Method of Image Classification with Optimized BP Neural Network by Genetic Algorithm

Optimization Study of BP Neural Network Based on Genetic Algorithm

Genetic algorithms and artificial neural network. (1) Training artificial neural network with extended genetic algorithms

Study on the fast learning algorithm of single-layer neural networks

Research and Application of an Optimized BP Neural Network Based on Adaptive Genetic Algorithm

Two-Times Training Algorithm of Neural Network Based on GA

Studies on Optimization Algorithms for Some Artificial Neural Networks Based on Genetic Algorithm (GA)