Base vector selection for kernel matching pursuit

Qing Li,Licheng Jiao

DOI: https://doi.org/10.1007/11811305_105

2006-01-01

Abstract:Kernel matching pursuit (KMP) is a relatively new learning algorithm to produce non-linear version of conventional supervised and unsupervised learning algorithm. However, it also contains some defects such as storage problem (in training process) and sparsity problem. In this paper, a new method is proposed to pre-select the base vectors from the original data according to vector correlation principle, which could greatly reduce the scale of the optimization problem and improve the sparsity of the solution. The method could capture the structure of the data space by approximating a basis of the subspace of the data; therefore, the statistical information of the training samples is preserved. In the paper, the deduction of mathematical process is given in details and the number of simulation results on artificial data and practical data has been done to validate the performance of base vector selection (BVS) algorithm. The experimental results show the combination of such algorithm with KMP can make great progress while leave the performance almost unchanged.