Adaptive inertia weights: an effective way to improve parameter estimation of hidden layer in stochastic configuration networks
Ying Han,Yuanhao Yu,Kun Li
DOI: https://doi.org/10.1007/s13042-024-02386-y
2024-10-06
International Journal of Machine Learning and Cybernetics
Abstract:Stochastic Configuration Networks (SCNs) is a new incremental neural network model, and its supervised mechanism plays a crucial role in ensuring the universal approximation ability. However, existing studies on SCNs have shown shortcomings, particularly the parameter estimation of the hidden layer during the model construction. To address this issue, many scholars have introduced additional mechanisms, which partially solve the aforementioned problems but increase the computational cost. Therefore, on the premise of maintaining the randomness and low complexity of SCNs, this paper aims to construct the model based on the direction of fastest decline in residuals, so as to obtain better regression performance. To achieve this goal, an input weight and bias configuration method based on adaptive inertia weights is proposed in this paper, named by Adaptive Weighted Stochastic Configuration Networks (AWSCNs). This method adaptively controls the node parameters to obtain smaller residual errors and improve model prediction ability. Additionally, the singular value decomposition algorithm is incorporated to compress the model and enhance the calculation speed. In this paper, the effectiveness and stability of the proposed method are evaluated using one approximation function and seven benchmark regression datasets. Experimental results demonstrate that AWSCNs not only get more ideal node parameters but also exhibit superior performance in terms of fast convergence, generalization capability, and running time.
computer science, artificial intelligence