Abstract:When dealing with complex nonlinear signals in intelligent system, by defining the inner product of two data vectors in the feature space, the kernel function can reflect the nonlinear mapping relationship between reproducing kernel Hilbert space (RKHS) and original data space. Therefore, by implementing classical linear adaptive filtering in RKHS space, the filtering operation can be expressed as a special relation of inner product with kernel function, which is referred to as “kernel trick.” As long as these algorithms can be expressed in the form of inner product, not only the convex least squares problem can be solved iteratively, but also the nonlinear adaptive filtering algorithms can be obtained, which have both general approximation characteristics and convexity. Therefore, the combination of kernel method and adaptive filtering algorithm is realized. On the other hand, since the Gram matrix is used, the dimension of kernel adaptive algorithm is determined by the number of data samples. When the number of observation sample point is increasing, the size of the state space increases exponentially with the growth of the dimension. Thereby, the kernel adaptive algorithm should solve the problem of online sparsification to avoid “curse of dimensionality.” As part II, based on online sparse kernel learning and the classical adaptive filtering algorithm, the kernel adaptive algorithms and online sparse algorithms are investigated in this paper. The main works of this paper have two aspects. First, combined with classical adaptive algorithm and kernel feature mapping, this paper investigates the basic concept of kernel adaptive algorithm and the realization mechanism of four kernel adaptive algorithm intensively: 1) kernel least mean squares; 2) kernel recursive least squares; 3) kernel affine projection algorithm; and 4) kernel principal component analysis. Second, in order to reduce the computational complexity, this paper studies some online sparse algorithms which include novel criterion, approximate linear dependency, sliding window, coherence criterion, surprise criterion, and so on. Finally, this paper summarizes the essence of existing conclusions of above algorithms and perspectives the future research direction.

Sequential Outlier Criterion for Sparsification of Online Adaptive Filtering

Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback

Outlier-resistant adaptive filtering based on sparse Bayesian learning

Research Progress on Intelligent System's Learning, Optimization, and Control--Part II: Online Sparse Kernel Adaptive Algorithm

Robust Kernel Recursive Adaptive Filtering Algorithms Based on M-estimate

Sparse Modeling-Based Sequential Ensemble Learning for Effective Outlier Detection in High-Dimensional Numeric Data.

Online Kernel Learning with a Near Optimal Sparsity Bound

Probability-Guaranteed Envelope-Constrained Filtering for Nonlinear Systems Subject to Measurement Outliers

Sequential sparse Bayesian learning with applications to system identification for damage assessment and recursive reconstruction of image sequences

Adaptive Sparse Confidence-Weighted Learning for Online Feature Selection.

Robust kernel adaptive filtering for nonlinear time series prediction

An Outlier Detection Scheme for Dynamical Sequential Datasets.

An Outlier-Robust Kalman Filter With Adaptive Selection of Elliptically Contoured Distributions

Outlier-Insensitive Kalman Filtering: Theory and Applications

Quantized criterion-based kernel recursive least squares adaptive filtering for time series prediction

A Novel Real-Time Filtering Method to General Nonlinear Filtering Problem Without Memory

Sequential online prediction in the presence of outliers and change points: An instant temporal structure learning approach

Online Feature Selection for Streaming Features Using Self-Adaption Sliding-Window Sampling

An Adaptive Online Smoother with Closed-Form Solutions and Information-Theoretic Lag Selection for Conditional Gaussian Nonlinear Systems

Threshold Optimization with a Small Number of Samples in Adaptive Information Filtering

Proportionate Recursive Maximum Correntropy Criterion Adaptive Filtering Algorithms and their Performance Analysis