Abstract:Breiman's random forest (RF) can be interpreted as an implicit kernel generator,where the ensuing proximity matrix represents the data-driven RF kernel. Kernel perspective on the RF has been used to develop a principled framework for theoretical investigation of its statistical properties. However, practical utility of the links between kernels and the RF has not been widely explored and systematically evaluated.Focus of our work is investigation of the interplay between kernel methods and the RF. We elucidate the performance and properties of the data driven RF kernels used by regularized linear models in a comprehensive simulation study comprising of continuous, binary and survival targets. We show that for continuous and survival targets, the RF kernels are competitive to RF in higher dimensional scenarios with larger number of noisy features. For the binary target, the RF kernel and RF exhibit comparable performance. As the RF kernel asymptotically converges to the Laplace kernel, we included it in our evaluation. For most simulation setups, the RF and RFkernel outperformed the Laplace kernel. Nevertheless, in some cases the Laplace kernel was competitive, showing its potential value for applications. We also provide the results from real life data sets for the regression, classification and survival to illustrate how these insights may be leveraged in practice.Finally, we discuss further extensions of the RF kernels in the context of interpretable prototype and landmarking classification, regression and survival. We outline future line of research for kernels furnished by Bayesian counterparts of the RF.

Improved convergence rates for some kernel random forest algorithms

Improved convergence rates for some kernel random forest algorithms

Random Forests and Kernel Methods

A simplified directional KeRF algorithm

Random Forest (RF) Kernel for Regression, Classification and Survival

Rates of convergence of randomized Kaczmarz algorithms in Hilbert spaces

Decentralized Kernel Ridge Regression Based on Data-Dependent Random Feature

Towards Convergence Rate Analysis of Random Forests for Classification

Improved kernel direct Fisher discriminant analysis and face recognition

Sharp Analysis of a Simple Model for Random Forests

Kernel Truncated Randomized Ridge Regression: Optimal Rates and Low Noise Acceleration

High-performance Kernel Machines with Implicit Distributed Optimization and Randomization

Learning Interpretable Characteristic Kernels via Decision Forests

Making Sense of Random Forest Probabilities: a Kernel Perspective

Statistical Optimality of Divide and Conquer Kernel-based Functional Linear Regression

Taming graph kernels with random features

Asymptotic Properties of High-Dimensional Random Forests

Kernel induced random survival forests

Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent

Improved Convergence Rate for Diffusion Probabilistic Models

Effect of dimensionality on convergence rates of kernel ridge regression estimator