Abstract:Single-hidden layer feed forward neural networks (SLFNs) are widely used in pattern classification problems, but a huge bottleneck encountered is the slow speed and poor performance of the traditional iterative gradient-based learning algorithms. Although the famous extreme learning machine (ELM) has successfully addressed the problems of slow convergence, it still has computational robustness problems brought by input weights and biases randomly assigned. Thus, in order to overcome the aforementioned problems, in this paper, a novel type neural network based on Gegenbauer orthogonal polynomials, termed as GNN, is constructed and investigated. This model could overcome the computational robustness problems of ELM, while still has comparable structural simplicity and approximation capability. Based on this, we propose a regularized weights direct determination (R-WDD) based on equality-constrained optimization to determine the optimal output weights. The R-WDD tends to minimize the empirical risks and structural risks of the network, thus to lower the risk of over fitting and improve the generalization ability. This leads us to a the final GNN with R-WDD, which is a unified learning mechanism for binary and multi-class classification problems. Finally, as is verified in the various comparison experiments, GNN with R-WDD tends to have comparable (or even better) generalization performances, computational scalability and efficiency, and classification robustness, compared to least square support vector machine (LS-SVM), ELM with Gaussian kernel.

A Learning Algorithm with Gaussian Regularizer for Kernel Neuron

Kernel Matrix Learning with a General Regularized Risk Functional Criterion

Learning Gaussian Networks

A new improved support vector machine: QGA-SVM

Sparse Training Procedure for Kernel Neuron

Kernel Neuron And Its Training Algorithm

Robust Regularized Kernel Regression.

Regularized Kernel Forms of Minimum Squared Error Method

Online Kernel Learning with a Near Optimal Sparsity Bound

Conjugate Gradients for Kernel Machines

A Gegenbauer Neural Network with Regularized Weights Direct Determination for Classification

Kernel Mse Algorithm: A Unified Framework for Kfd, Ls-Svm and Krr

Nonlinear Pocket Algorithm with Kernels

Gaussian Kernel Variance For an Adaptive Learning Method on Signals Over Graphs

An Adaptive Gaussian Kernel for Support Vector Machine

Stochastic three-term conjugate gradient method with variance technique for non-convex learning

Multikernel semiparametric linear programming support vector regression

Machine learning-based system reliability analysis with Gaussian Process Regression

Pairwise Ranking with Gaussian Kernel

A Unified Kernel for Neural Network Learning

Gauss-like Logarithmic Kernel Function to improve the performance of kernel machines on the small datasets