Abstract:This work addresses two main issues of the standard Kernel Entropy Component Analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of by variance as in Kernel Principal Components Analysis. In this work, we propose an extension of the KECA method, named Optimized KECA (OKECA), that directly extracts the optimal features retaining most of the data entropy by means of compacting the information in very few features (often in just one or two). The proposed method produces features which have higher expressive power. In particular, it is based on the Independent Component Analysis (ICA) framework, and introduces an extra rotation to the eigen-decomposition, which is optimized via gradient ascent search. This maximum entropy preservation suggests that OKECA features are more efficient than KECA features for density estimation. In addition, a critical issue in both methods is the selection of the kernel parameter since it critically affects the resulting performance. Here we analyze the most common kernel length-scale selection criteria. Results of both methods are illustrated in different synthetic and real problems. Results show that 1) OKECA returns projections with more expressive power than KECA, 2) the most successful rule for estimating the kernel parameter is based on maximum likelihood, and 3) OKECA is more robust to the selection of the length-scale parameter in kernel density estimation.

Indefinite Kernel Entropy Component Analysis

Optimized Kernel Entropy Components

Improved KECA for Process Monitoring Using Ensemble Learning and Bayesian Inference

L1-norm-based Kernel Entropy Components

Sparse Kernel Entropy Component Analysis for Dimensionality Reduction of Biomedical Data

Adaptive Kpca Modeling of Nonlinear Systems

Cross-validatory Framework for Optimal Parameter Estimation of KPCA and KPLS Models

Kernel Minimum Error Entropy Algorithm

Measures of Entropy From Data Using Infinitely Divisible Kernels

Adaptive quasiconformal kernel discriminant analysis

Fault Detection in Nonlinear Chemical Processes Based on Kernel Entropy Component Analysis and Angular Structure

Human Face Recognition using Gabor based Kernel Entropy Component Analysis

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

An Infinite Dimensional Analysis of Kernel Principal Components

An Efficient Reformative Kernel Discriminant Analysis for Face Recognition

Classification algorithm of hyperspectral images based on kernel entropy analysis

Kernel Mean Embedding of Probability Measures and its Applications to Functional Data Analysis

Positive Definite Kernels, Algorithms, Frames, and Approximations

A Face Recognition approach based on entropy estimate of the nonlinear DCT features in the Logarithm Domain together with Kernel Entropy Component Analysis

Conditional mean embeddings and optimal feature selection via positive definite kernels

Kinetic Energy Plus Penalty Functions for Sparse Estimation