Online Component Analysis, Architectures and Applications.
Joao B. O. Souza Filho,Lan-Da Van,Tzyy-Ping Jung,Paulo S. R. Diniz
DOI: https://doi.org/10.1561/2000000112
2022-01-01
Foundations and Trends in Signal Processing
Abstract:This monograph deals with principal component analysis (PCA), kernel component analysis (KPCA), and independent component analysis (ICA), highlighting their applications to streaming-data implementations. The basic concepts related to PCA, KPCA, and ICA are widely available in the literature; however, very few texts deal with their practical implementation in computationally limited resources. The presentation tries to emphasize the current solutions considering possible constraints in power consumption and desirable computational complexity. For instance, there are good examples in biomedical engineering applications where tools like PCA and ICA can sort out the human body’s activities. For example, it is possible to remove noise and undesirable artifacts from a target signal such as EEG and ECG, among others. In turn, KPCA may be a valuable resource for non-linear image denoising. Nonetheless, many current solutions rely on batch processing implemented in general-purpose computing resources. In general terms, PCA consists of a sequence of uncorrelated data projections ordered according to their variances and employing mutually orthogonal directions. PCA is mighty in extracting hidden linear structures in high-dimension datasets. The standard PCA implementation computes the eigenvectors of the data-covariance matrix, retaining those directions to which the data exhibit the highest projection variances. This concept can be extended to the so-called Kernel PCA, wherein the data instances are implicitly mapped into a high-dimensional feature space via some non-linear transform, typically unknown. Conversely, ICA strengthens the PCA maximization variance approach by imposing the strict premise of mutual independence on the resulting projections. In fact, ICA comes to rescue the traditional tools when one aims at assessing non-Gaussian sources from data, often not available for direct measurement. Frequently, ICA and KPCA are more powerful tools for solving challenging tasks than PCA since they exploit high-order statistics from data. All these methods require some simplifications to allow a simple online implementation when coping with streaming data. This monograph describes some state-of-the-art solutions for PCA, KPCA, and ICA, emphasizing their online deployments. Many online PCA and, more recently, KPCA techniques were proposed based on Hebbian learning rules and fixed-point iterative equations. Notably, online KPCA solutions also include data selection strategies to define a compact dictionary over which the kernel components are expanded. The complexity of these dictionaries is controlled by simply setting a single hyperparameter. In both cases, the online extensions proposed rely on simple equations, can track nonstationary environments, and require reduced storage, enabling its use in real-time applications operating in low-cost embedded hardware. This monograph discusses the state-of-the-art online PCA and KPCA techniques in a unified and principled manner, presenting solutions that achieve a higher convergence speed and accuracy in many applications, particularly image processing. Besides, this work also explains how to remove various artifacts from data records based on blind source separation (BSS) by ICA, splitting feature identification from feature separation. Herein, three FastICA online hardware architectures and implementation for biomedical signal processing are addressed. The main features are summarized as follows: 1) energy-efficient FastICA using the early determination scheme; 2) cost-effective variable-channel FastICA using the Gram-Schmidt-based whitening algorithm; and 3) moving-window-based online FastICA algorithm with limited memory. The post-layout simulation results with artificial and EEG data validate the design concepts. In summary, this monograph presents the leading algorithmic solutions for PCA, KPCA, ICA, Iterative PCA, Online KPCA, and Online ICA, focusing on approaches amenable to process streaming signals. Furthermore, it provides some insights into how to choose the right solution for practical systems. Along the way, some implementation examples are provided in a variety of areas.