Abstract:The random Fourier features (RFFs) method is a powerful and popular technique in kernel approximation for scalability of kernel methods. The theoretical foundation of RFFs is based on the Bochner theorem (Bochner in Harmonic Analysis and the Theory of Probability, University of California Press, 1995) that relates symmetric, positive definite (PD) functions to probability measures. This condition naturally excludes asymmetric functions with a wide range applications in practice, e.g., directed graphs, conditional probability, and asymmetric kernels. Nevertheless, understanding asymmetric functions (kernels) and its scalability via RFFs is unclear both theoretically and empirically. In this paper, we introduce a complex measure with the real and imaginary parts corresponding to four finite positive measures, which expands the application scope of the Bochner theorem. By doing so, this framework allows for handling classical symmetric, PD kernels via one positive measure; symmetric, non-positive definite kernels via signed measures; and asymmetric kernels via complex measures, thereby unifying them into a general framework by RFFs, named AsK-RFFs. Such approximation scheme via complex measures enjoys theoretical guarantees in the perspective of uniform convergence. In algorithmic implementation, to speed up the kernel approximation process, which is expensive due to the calculation of total masses, we propose a subset-based fast estimation method. This method focuses on optimizing total masses within a sub-training set, effectively transforming the numerical integration for total masses into quadratic programming in three-dimension with low time complexity. AsK-RFFs provides two explicit feature mappings to approximate an asymmetric kernel. These mappings can be utilized in classifiers within the framework of AsK-LS (He et al. in IEEE Trans Pattern Anal Machine Intell 45(8):10044–10054), fulfilling the purpose of using asymmetric kernels in machine learning applications. Our AsK-RFFs method is empirically validated on several typical large-scale datasets and achieves promising kernel approximation performance, which demonstrates the effectiveness of AsK-RFFs.

All Random Features Representations are Equivalent

All Random Features Representations are Equivalent

Towards Unbiased Random Features with Lower Variance for Stationary Indefinite Kernels

Random Fourier Feature Kernel Recursive Least Squares.

A Double-Variational Bayesian Framework in Random Fourier Features for Indefinite Kernels

Random Fourier Features Via Fast Surrogate Leverage Weighted Sampling

Random Features Approximation for Control-Affine Systems

Sample-Efficient Linear Representation Learning from Non-IID Non-Isotropic Data

Improved Random Features for Dot Product Kernels

Orthogonal Random Features: Explicit Forms and Sharp Inequalities

Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?

Random Representations Outperform Online Continually Learned Representations

Random fourier features for asymmetric kernels

On the Approximation Properties of Random ReLU Features

On the Power of Randomization in Fair Classification and Representation

Shallow Representation is Deep: Learning Uncertainty-aware and Worst-case Random Feature Dynamics

Universal approximation property of Banach space-valued random feature models including random neural networks

Simplex Random Features

Learning unbiased features

Feasibility of random basis function approximators for modeling and control

Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison.

All Random Features Representations are Equivalent

All Random Features Representations are Equivalent

Towards Unbiased Random Features with Lower Variance for Stationary Indefinite Kernels

Random Fourier Feature Kernel Recursive Least Squares.

A Double-Variational Bayesian Framework in Random Fourier Features for Indefinite Kernels

Random Fourier Features Via Fast Surrogate Leverage Weighted Sampling

Random Features Approximation for Control-Affine Systems

Sample-Efficient Linear Representation Learning from Non-IID Non-Isotropic Data

Improved Random Features for Dot Product Kernels

Orthogonal Random Features: Explicit Forms and Sharp Inequalities

Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?

Random Representations Outperform Online Continually Learned Representations

Random fourier features for asymmetric kernels

On the Approximation Properties of Random ReLU Features

On the Power of Randomization in Fair Classification and Representation

Shallow Representation is Deep: Learning Uncertainty-aware and Worst-case Random Feature Dynamics

Universal approximation property of Banach space-valued random feature models including random neural networks

Simplex Random Features

Learning unbiased features

Feasibility of random basis function approximators for modeling and control

Nystr&ouml;m Method vs Random Fourier Features: A Theoretical and Empirical Comparison.

Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison.