Abstract:A simple yet effective architectural design of radial basis function neural networks (RBFNN) makes them amongst the most popular conventional neural networks. The current generation of radial basis function neural network is equipped with multiple kernels which provide significant performance benefits compared to the previous generation using only a single kernel. In existing multi-kernel RBF algorithms, multi-kernel is formed by the convex combination of the base/primary kernels. In this paper, we propose a novel multi-kernel RBFNN in which every base kernel has its own (local) weight. This novel flexibility in the network provides better performance such as faster convergence rate, better local minima and resilience against stucking in poor local minima. These performance gains are achieved at a competitive computational complexity compared to the contemporary multi-kernel RBF algorithms. The proposed algorithm is thoroughly analysed for performance gain using mathematical and graphical illustrations and also evaluated on three different types of problems namely: (i) pattern classification, (ii) system identification and (iii) function approximation. Empirical results clearly show the superiority of the proposed algorithm compared to the existing state-of-the-art multi-kernel approaches.
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is the insufficient generalization ability of the existing Radial Basis Function Neural Network (RBFNN) when dealing with multi - source heterogeneous data. Specifically:
1. **Single - kernel Limitations**: Traditional RBFNN usually uses only a single type of kernel function, which limits its generalization ability, especially when facing complex practical learning problems that often involve data from multiple different sources.
2. **Limitations of Multi - kernel Fusion Methods**: Existing multi - kernel RBFNN algorithms mainly linearly combine multiple kernel functions through convex combination, and all participating kernels share a set of weights or coefficients. This conservative choice makes these methods perform poorly in some cases and easily fall into local optimal solutions.
### Solutions Proposed in the Paper
To solve the above problems, the paper proposes a new multi - kernel RBFNN architecture - the Coordinated RBFNN (Co - RBFNN). The main innovations of this method include:
- **Independent Local Weights**: Each participating kernel function has its own independent weights (local weights), instead of sharing a set of weights as in traditional methods. This can improve the flexibility and performance of the model, such as faster convergence speed, better local minima, and stronger resistance to local optimal solutions.
- **Mixed Weights in Matrix Form**: By introducing mixed weights in matrix form, each kernel function can learn independently, thus obtaining better performance in most cases.
### Mathematical Representation
To understand this method more clearly, the following are the key formulas:
- Output formula for multi - kernel fusion:
\[
y(n)=\sum_{k, l} w_{k, l}(n) \varphi_{l, k}(x(n), m_{k})+b(n)
\]
where \(k = 1,2,\ldots,K\), \(l\in\{g, c\}\), \(w_{k, l}(n)=w_{k}(n)\alpha_{l, k}(n)\) is the weight of the \(l\) - th participating kernel in the \(k\) - th multi - kernel, and \(x(n)\) is the sample of the \(n\) - th iteration obtained from the input data set \(X\).
- Weight update rules:
\[
w_{k, l}(n + 1)=w_{k, l}(n)+\eta e(n)\varphi_{l, k}(x(n), m_{k})
\]
\[
b(n + 1)=b(n)+\eta e(n)
\]
where \(\eta\) is the learning rate and \(e(n)\) is the instantaneous error.
### Experimental Results
The paper verifies the effectiveness of the proposed method through three different types of problems: pattern classification, system identification, and function approximation. The experimental results show that, compared with the existing multi - kernel RBFNN algorithms, the proposed Co - RBFNN has significant advantages in performance.
In conclusion, by introducing independent local weights and mixed weights in matrix form, this paper effectively solves the problem of insufficient generalization ability of traditional multi - kernel RBFNN when dealing with complex multi - source data, and provides faster convergence speed and better performance.