Abstract:Neural networks (NN) achieve remarkable results in various tasks, but lack key characteristics: interpretability, support for categorical features, and lightweight implementations suitable for edge devices. While ongoing efforts aim to address these challenges, Gradient Boosting Trees (GBT) inherently meet these requirements. As a result, GBTs have become the go-to method for supervised learning tasks in many real-world applications and competitions. However, their application in online learning scenarios, notably in reinforcement learning (RL), has been limited. In this work, we bridge this gap by introducing Gradient-Boosting RL (GBRL), a framework that extends the advantages of GBT to the RL domain. Using the GBRL framework, we implement various actor-critic algorithms and compare their performance with their NN counterparts. Inspired by shared backbones in NN we introduce a tree-sharing approach for policy and value functions with distinct learning rates, enhancing learning efficiency over millions of interactions. GBRL achieves competitive performance across a diverse array of tasks, excelling in domains with structured or categorical features. Additionally, we present a high-performance, GPU-accelerated implementation that integrates seamlessly with widely-used RL libraries (available at <a class="link-external link-https" href="https://github.com/NVlabs/gbrl" rel="external noopener nofollow">this https URL</a>). GBRL expands the toolkit for RL practitioners, demonstrating the viability and promise of GBT within the RL paradigm, particularly in domains characterized by structured or categorical features.

Functional Gradient Boosting based on Residual Network Perception

Residual Networks Behave Like Boosting Algorithms

Learning Deep ResNet Blocks Sequentially using Boosting Theory

Representational Gradient Boosting: Backpropagation in the Space of Functions

Optimization by gradient boosting

A Deep Gradient Boosting Network for Optic Disc and Cup Segmentation

Gradient Boosting Neural Networks: GrowNet

A Gradient Boosting Approach for Training Convolutional and Deep Neural Networks

Gradient and Newton boosting for classification and regression

Gradient Boosting Reinforcement Learning

A New Formulation of Gradient Boosting

Gradient Descent Optimizes Normalization-Free ResNets.

Tree-based boosting with functional data

Boosting Residual Networks with Group Knowledge

Multi-Layered Gradient Boosting Decision Trees.

FedGBF: An efficient vertical federated learning framework via gradient boosting and bagging

Sequential Training of Neural Networks with Gradient Boosting

Deep Learning for Multi-Output Regression Using Gradient Boosting

Gradient Networks

Gradient Boosting Mapping for Dimensionality Reduction and Feature Extraction

QBoost: Predicting quantiles with boosting for regression and binary classification