Improved convergence rates for some kernel random forest algorithms

Isidoros Iakovidis,Nicola Arcozzi

2023-10-11

Abstract:Random forests are notable learning algorithms first introduced by Breinman in 2001, they are widely used for classification and regression tasks and their mathematical properties are under ongoing research. We consider a specific class of random forest algorithms related to kernel methods, the so-called KeRF (Kernel Random Forests.) In particular, we investigate thoroughly two explicit algorithms, designed independently of the data set, the centered KeRF and the uniform KeRF. In the present article, we provide an improvement in the rate of convergence for both algorithms and we explore the related reproducing kernel Hilbert space defined by the explicit kernel of the centered random forest.

Statistics Theory

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper primarily explores the issue of improving the convergence rates of certain kernel-based random forest algorithms (Kernel Random Forests, KeRF).

#### Summary of Main Content:
1. **Research Background**:
- Since its introduction by Breiman in 2001, the random forest algorithm has shown excellent performance in classification and regression tasks, with high accuracy.
- Despite its good performance in practical applications, its mathematical properties remain an active area of research.

2. **Research Object**:
- The researchers focus on specific types of random forest algorithms related to kernel methods, namely KeRF.
- Specifically, the paper studies two explicit algorithms in detail: centered KeRF and uniform KeRF.

3. **Main Contributions**:
- Provides improved convergence rates for the centered KeRF and uniform KeRF algorithms.
- Explores the related Reproducing Kernel Hilbert Space (RKHS) defined by the explicit kernel of the centered random forest.

4. **Specific Results**:
- For the centered KeRF algorithm, the paper provides an improved estimate of the convergence rate:
\[
E(\tilde{m}_{\text{cen}}^{\infty, n}(x) - m(x))^2 \leq \tilde{C} n^{-\frac{1}{1+d\log 2}} (\log n).
\]
- For the uniform KeRF algorithm, the paper also provides an improved estimate of the convergence rate:
\[
E(\tilde{m}_{\text{un}}^{\infty, n}(x) - m(x))^2 \leq \tilde{C} n^{-\frac{1}{1+\frac{3}{2}d\log 2}} (\log n).
\]

5. **Experimental Validation**:
- Provides numerical experiments comparing the L2 error under different tree depths, validating the effectiveness of the theoretical results.

6. **Further Analysis**:
- Analyzes the reproducing kernel used in the centered KeRF algorithm and its Fourier transform on a finite Abelian group, obtaining related expressions and function properties.

Through these efforts, the paper significantly improves the convergence speed of KeRF algorithms and provides a new theoretical foundation for subsequent research.