An experimental evaluation of large scale GBDT systems

FuFangeheng,JiangJiawei,ShaoYingxia,CuiBin

2019-01-01

Abstract:Gradient boosting decision tree (GBDT) is a widely-used machine learning algorithm in both data analytic competitions and real-world industrial applications. Further, driven by the rapid increase i...

What problem does this paper attempt to address?

An Experimental Evaluation of Large Scale GBDT Systems

Fangcheng Fu,Jiawei Jiang,Yingxia Shao,Bin Cui

DOI: https://doi.org/10.14778/3342263.3342273

IF: 2.5

2019-01-01

Proceedings of the VLDB Endowment

Abstract:Gradient boosting decision tree (GBDT) is a widely-used machine learning algorithm in both data analytic competitions and real-world industrial applications. Further, driven by the rapid increase in data volume, efforts have been made to train GBDT in a distributed setting to support large-scale workloads. However, we find it surprising that the existing systems manage the training dataset in different ways, but none of them have studied the impact of data management. To that end, this paper aims to study the pros and cons of different data management methods regarding the performance of distributed GBDT. We first introduce a quadrant categorization of data management policies based on data partitioning and data storage. Then we conduct an in-depth systematic analysis and summarize the advantageous scenarios of the quadrants. Based on the analysis, we further propose a novel distributed GBDT system named Vero, which adopts the unexplored composition of vertical partitioning and row-store and suits for many large-scale cases. To validate our analysis empirically, we implement different quadrants in the same code base and compare them under extensive workloads, and finally compare Vero with other state-of-the-art systems over a wide range of datasets. Our theoretical and experimental results provide a guideline on choosing a proper data management policy for a given workload.
DimBoost

Jiawei Jiang,Bin Cui,Ce Zhang,Fangcheng Fu

DOI: https://doi.org/10.1145/3183713.3196892

2018-01-01

Abstract:Gradient boosting decision tree (GBDT) is one of the most popular machine learning models widely used in both academia and industry. Although GBDT has been widely supported by existing systems such as XGBoost, LightGBM, and MLlib, one system bottleneck appears when the dimensionality of the data becomes high. As a result, when we tried to support our industrial partner on datasets of the dimension up to 330K, we observed suboptimal performance for all these aforementioned systems. In this paper, we ask "Can we build a scalable GBDT training system whose performance scales better with respect to dimensionality of the data?"
Challenges and Opportunities of Building Fast GBDT Systems.

Zeyi Wen,Qinbin Li,Bingsheng He,Bin Cui

DOI: https://doi.org/10.24963/ijcai.2021/632

2021-01-01

Abstract:In the last few years, Gradient Boosting Decision Trees (GBDTs) have been widely used in various applications such as online advertising and spam filtering. However, GBDT training is often a key performance bottleneck for such data science pipelines, especially for training a large number of deep trees on large data sets. Thus, many parallel and distributed GBDT systems have been researched and developed to accelerate the training process. In this survey paper, we review the recent GBDT systems with respect to accelerations with emerging hardware as well as cluster computing, and compare the advantages and disadvantages of the existing implementations. Finally, we present the research opportunities and challenges in designing fast next generation GBDT systems.
Gradient Boosting With Piece-Wise Linear Regression Trees

Yu Shi,Jian Li,Zhize Li

DOI: https://doi.org/10.48550/arXiv.1802.05640

2019-06-26

Abstract:Gradient Boosted Decision Trees (GBDT) is a very successful ensemble learning algorithm widely used across a variety of applications. Recently, several variants of GBDT training algorithms and implementations have been designed and heavily optimized in some very popular open sourced toolkits including XGBoost, LightGBM and CatBoost. In this paper, we show that both the accuracy and efficiency of GBDT can be further enhanced by using more complex base learners. Specifically, we extend gradient boosting to use piecewise linear regression trees (PL Trees), instead of piecewise constant regression trees, as base learners. We show that PL Trees can accelerate convergence of GBDT and improve the accuracy. We also propose some optimization tricks to substantially reduce the training time of PL Trees, with little sacrifice of accuracy. Moreover, we propose several implementation techniques to speedup our algorithm on modern computer architectures with powerful Single Instruction Multiple Data (SIMD) parallelism. The experimental results show that GBDT with PL Trees can provide very competitive testing accuracy with comparable or less training time.

Machine Learning
Identifying Transportation Modes Using Gradient Boosting Decision Tree

Xin Fu,Dong Wang,Hengcai Zhang

DOI: https://doi.org/10.1007/978-3-319-95933-7_63

2018-01-01

Abstract:Identifying the transportation modes could be applicable to many applications including personalized recommendation, transportation planning. The existing studies had not fully considered the impact of geographical information. In this paper, we propose a novel approach to detect transportation modes from massive trajectories using Gradient Boosting Decision Tree (GBDT), which adopted and estimated the impact of geographical information to achieve a better performance. In the experiments, we conduct the performance evaluation using the Geolife dataset which collected by 182 users over five years. The dataset contains 8347 trajectories with transportation mode such as driving, taking a bus, riding a bike and walking. 60% of trajectories are randomly chosen as training dataset, and then we tested on the remaining dataset. The experimental results showed that our proposed approach considering geographical information by using gradient boosting decision tree method achieve the precision of 84%, with the maximum increase of 6.83% to the traditional identifying transportation modes method. In addition, the geographical information contributed over 12% to improve the precision of recognition.
Benchmarking and Optimization of Gradient Boosting Decision Tree Algorithms

Andreea Anghel,Nikolaos Papandreou,Thomas Parnell,Alessandro De Palma,Haralampos Pozidis

DOI: https://doi.org/10.48550/arXiv.1809.04559

2019-01-17

Abstract:Gradient boosting decision trees (GBDTs) have seen widespread adoption in academia, industry and competitive data science due to their state-of-the-art performance in many machine learning tasks. One relative downside to these models is the large number of hyper-parameters that they expose to the end-user. To maximize the predictive power of GBDT models, one must either manually tune the hyper-parameters, or utilize automated techniques such as those based on Bayesian optimization. Both of these approaches are time-consuming since they involve repeatably training the model for different sets of hyper-parameters. A number of software GBDT packages have started to offer GPU acceleration which can help to alleviate this problem. In this paper, we consider three such packages: XGBoost, LightGBM and Catboost. Firstly, we evaluate the performance of the GPU acceleration provided by these packages using large-scale datasets with varying shapes, sparsities and learning tasks. Then, we compare the packages in the context of hyper-parameter optimization, both in terms of how quickly each package converges to a good validation score, and in terms of generalization performance.

Machine Learning
SecureBoost+: Large Scale and High-Performance Vertical Federated Gradient Boosting Decision Tree

Tao Fan,Weijing Chen,Guoqiang Ma,Yan Kang,Lixin Fan,Qiang Yang

2024-06-19

Abstract:Gradient boosting decision tree (GBDT) is an ensemble machine learning algorithm, which is widely used in industry, due to its good performance and easy interpretation. Due to the problem of data isolation and the requirement of privacy, many works try to use vertical federated learning to train machine learning models collaboratively with privacy guarantees between different data owners. SecureBoost is one of the most popular vertical federated learning algorithms for GBDT. However, in order to achieve privacy preservation, SecureBoost involves complex training procedures and time-consuming cryptography operations. This causes SecureBoost to be slow to train and does not scale to large scale data. In this work, we propose SecureBoost+, a large-scale and high-performance vertical federated gradient boosting decision tree framework. SecureBoost+ is secure in the semi-honest model, which is the same as SecureBoost. SecureBoost+ can be scaled up to tens of millions of data samples easily. SecureBoost+ achieves high performance through several novel optimizations for SecureBoost, including ciphertext operation optimization, the introduction of new training mechanisms, and multi-classification training optimization. The experimental results show that SecureBoost+ is 6-35x faster than SecureBoost, but with the same accuracy and can be scaled up to tens of millions of data samples and thousands of feature dimensions.

Machine Learning,Artificial Intelligence
Gradient Boosted Binary Histogram Ensemble for Large-scale Regression

Hanyuan Hang,Tao Huang,Yuchao Cai,Hanfang Yang,Zhouchen Lin

DOI: https://doi.org/10.48550/arXiv.2106.01986

2021-06-04

Abstract:In this paper, we propose a gradient boosting algorithm for large-scale regression problems called \textit{Gradient Boosted Binary Histogram Ensemble} (GBBHE) based on binary histogram partition and ensemble learning. From the theoretical perspective, by assuming the Hölder continuity of the target function, we establish the statistical convergence rate of GBBHE in the space $C^{0,\alpha}$ and $C^{1,0}$, where a lower bound of the convergence rate for the base learner demonstrates the advantage of boosting. Moreover, in the space $C^{1,0}$, we prove that the number of iterations to achieve the fast convergence rate can be reduced by using ensemble regressor as the base learner, which improves the computational efficiency. In the experiments, compared with other state-of-the-art algorithms such as gradient boosted regression tree (GBRT), Breiman's forest, and kernel-based methods, our GBBHE algorithm shows promising performance with less running time on large-scale datasets.

Machine Learning
TencentBoost: A Gradient Boosting Tree System with Parameter Server

Jie Jiang,Jiawei Jiang,Bin Cui,Ce Zhang

DOI: https://doi.org/10.1109/icde.2017.87

2017-01-01

Abstract:Gradient boosting tree (GBT), a widely used machine learning algorithm, achieves state-of-the-art performance in academia, industry, and data analytics competitions. Although existing scalable systems which implement GBT, such as XGBoost and MLlib, perform well for datasets with medium-dimensional features, they can suffer performance degradation for many industrial applications where the trained datasets contain highdimensional features. The performance degradation derives from their inefficient mechanisms for model aggregation-either mapreduce or all-reduce. To address this high-dimensional problem, we propose a scalable execution plan using the parameter server architecture to facilitate the model aggregation. Further, we introduce a sparse-pull method and an efficient index structure to increase the processing speed. We implement a GBT system, namely TencentBoost, in the production cluster of Tencent Inc. The empirical results show that our system is 2-20× faster than existing platforms.
Label Aggregation of Gradient Boosting Decision Trees

X. C. Xiang,H. X. Zhang,S. T. Xia

DOI: https://doi.org/10.1145/3421558.3421581

2020-01-01

Abstract:Gradient boosting decision tree (GBDT) is one of the most widely used ensemble learning methods in both academia and industry. The core idea of GBDT is to consecutively fit new base learners to the residual errors between true and predicted values. However, GBDT is sensitive to noise in the data, especially label noise, which is very common in real-world data. Besides, due to the strong learning ability of GBDT, it is not easy to address this problem with simple ensemble method such as bagging. Thus we propose a new ensemble model of GBDT, called label aggregation of GBDTs (LA-GBDTs). The key points of our method lie on the random feature subspace and label aggregation, which lead to computational efficiency and robustness to noise. Experiments on several benchmark datasets show that label aggregation of GBDTs outperforms GBDT algorithms consistently in terms of accuracy. In addition, LA-GBDTs work well on datasets with a certain degree of label noise.
Poster: gbdt-rs: Fast and Trustworthy Gradient Boosting Decision Tree

Tianyi Li,Tongxin Li,Yu Ding,Yulong Zhang,Tao Wei,Xinhui Han

2019-01-01

Abstract:For data analysis services, the capability of preserving user privacy has become an eagerly demanded trait. However, most existing approaches are either unsound to close some major information leakage chanels or prohibitively expensive for practical deployment. We propose gbdt-rs , a secure and fast implementation of the gradient boosting decision tree algorithm, which is widely used in data mining and machine learning tasks. gbdt-rs is designed to fit traditional computing platforms as well as hardware-enforced trusted environments, i.e., Intel SGX secure enclaves. gbdt-rs is coded in the Rust programming language to exterminate memory errors, while its performance is barely sacrificed due to sophisticatedly optimized memory access patterns. Our experiments show that gbdt-rs can provide up to 10x speedup on inference tasks compared to XGBoost, a reputed C++ implementation of the same algorithm. In addition, gbdt-rs is highly auditable due to its relatively small code base.
HarpGBDT: Optimizing Gradient Boosting Decision Tree for Parallel Efficiency

Bo Peng,Langshi Chen,Jiayu Li,Miao Jiang,Selahattin Akkas,Egor Smirnov,Ruslan Israfilov,Sergey Khekhnev,Andrey Nikolaev,Judy Qiu

DOI: https://doi.org/10.1109/cluster.2019.8890990

2019-01-01

Abstract:Gradient Boosting Decision Tree (GBDT) is a widely used machine learning algorithm, whose training involves both irregular computation and random memory access and is challenging for system optimizations. In this paper, we conduct a comprehensive performance analysis of two state-of-the-art systems, XGBoost and LightGBM. They represent two typical parallel implementations for GBDT; one is data parallel and the other one is parallel over features. Substantial thread synchronization overhead, as well as the inefficiency of random memory access, is identified. We propose HarpGBDT, a new GBDT system designed from the perspective of parallel efficiency optimization. Firstly, we adopt a new tree growth method that selects the top K candidates of tree nodes to enable the use of more levels of parallelism without sacrificing the algorithm's accuracy. Secondly, we organize the training data and model data in blocks and propose a block-wise approach as a general model that enables the exploration of various parallelism options. Thirdly, we propose a mixed mode to utilize the advantages of a different mode of parallelism in different phases of training. By changing the configuration of the block size and parallel mode, HarpGBDT is able to attain better parallel efficiency. By extensive experiments on four datasets with different statistical characteristics on the Intel(R) Xeon(R) E5-2699 server, HarpGBDT on average performs 8x faster than XGBoost and 2.6x faster than LightGBM.
Unbiased Gradient Boosting Decision Tree with Unbiased Feature Importance

Zheyu Zhang,Tianping Zhang,Jian Li

2023-05-18

Abstract:Gradient Boosting Decision Tree (GBDT) has achieved remarkable success in a wide variety of applications. The split finding algorithm, which determines the tree construction process, is one of the most crucial components of GBDT. However, the split finding algorithm has long been criticized for its bias towards features with a large number of potential splits. This bias introduces severe interpretability and overfitting issues in GBDT. To this end, we provide a fine-grained analysis of bias in GBDT and demonstrate that the bias originates from 1) the systematic bias in the gain estimation of each split and 2) the bias in the split finding algorithm resulting from the use of the same data to evaluate the split improvement and determine the best split. Based on the analysis, we propose unbiased gain, a new unbiased measurement of gain importance using out-of-bag samples. Moreover, we incorporate the unbiased property into the split finding algorithm and develop UnbiasedGBM to solve the overfitting issue of GBDT. We assess the performance of UnbiasedGBM and unbiased gain in a large-scale empirical study comprising 60 datasets and show that: 1) UnbiasedGBM exhibits better performance than popular GBDT implementations such as LightGBM, XGBoost, and Catboost on average on the 60 datasets and 2) unbiased gain achieves better average performance in feature selection than popular feature importance methods. The codes are available at <a class="link-external link-https" href="https://github.com/ZheyuAqaZhang/UnbiasedGBM" rel="external noopener nofollow">this https URL</a>.

Machine Learning
Quantized Training of Gradient Boosting Decision Trees

Yu Shi,Guolin Ke,Zhuoming Chen,Shuxin Zheng,Tie-Yan Liu

DOI: https://doi.org/10.48550/arXiv.2207.09682

IF: 5.414

2022-07-20

Machine Learning

Abstract:Recent years have witnessed significant success in Gradient Boosting Decision Trees (GBDT) for a wide range of machine learning applications. Generally, a consensus about GBDT's training algorithms is gradients and statistics are computed based on high-precision floating points. In this paper, we investigate an essentially important question which has been largely ignored by the previous literature: how many bits are needed for representing gradients in training GBDT? To solve this mystery, we propose to quantize all the high-precision gradients in a very simple yet effective way in the GBDT's training algorithm. Surprisingly, both our theoretical analysis and empirical studies show that the necessary precisions of gradients without hurting any performance can be quite low, e.g., 2 or 3 bits. With low-precision gradients, most arithmetic operations in GBDT training can be replaced by integer operations of 8, 16, or 32 bits. Promisingly, these findings may pave the way for much more efficient training of GBDT from several aspects: (1) speeding up the computation of gradient statistics in histograms; (2) compressing the communication cost of high-precision statistical information during distributed training; (3) the inspiration of utilization and development of hardware architectures which well support low-precision computation for GBDT training. Benchmarked on CPUs, GPUs, and distributed clusters, we observe up to 2$\times$ speedup of our simple quantization strategy compared with SOTA GBDT systems on extensive datasets, demonstrating the effectiveness and potential of the low-precision training of GBDT. The code will be released to the official repository of LightGBM.
GBDT-MO: Gradient Boosted Decision Trees for Multiple Outputs

Zhendong Zhang,Cheolkon Jung

DOI: https://doi.org/10.48550/arXiv.1909.04373

2019-12-28

Abstract:Gradient boosted decision trees (GBDTs) are widely used in machine learning, and the output of current GBDT implementations is a single variable. When there are multiple outputs, GBDT constructs multiple trees corresponding to the output variables. The correlations between variables are ignored by such a strategy causing redundancy of the learned tree structures. In this paper, we propose a general method to learn GBDT for multiple outputs, called GBDT-MO. Each leaf of GBDT-MO constructs predictions of all variables or a subset of automatically selected variables. This is achieved by considering the summation of objective gains over all output variables. Moreover, we extend histogram approximation into multiple output case to speed up the training process. Various experiments on synthetic and real-world datasets verify that GBDT-MO achieves outstanding performance in terms of both accuracy and training speed. Our codes are available on-line.

Computer Vision and Pattern Recognition,Machine Learning
Improving GBDT Performance on Imbalanced Datasets: An Empirical Study of Class-Balanced Loss Functions

Jiaqi Luo,Yuan Yuan,Shixin Xu

2024-07-19

Abstract:Class imbalance remains a significant challenge in machine learning, particularly for tabular data classification tasks. While Gradient Boosting Decision Trees (GBDT) models have proven highly effective for such tasks, their performance can be compromised when dealing with imbalanced datasets. This paper presents the first comprehensive study on adapting class-balanced loss functions to three GBDT algorithms across various tabular classification tasks, including binary, multi-class, and multi-label classification. We conduct extensive experiments on multiple datasets to evaluate the impact of class-balanced losses on different GBDT models, establishing a valuable benchmark. Our results demonstrate the potential of class-balanced loss functions to enhance GBDT performance on imbalanced datasets, offering a robust approach for practitioners facing class imbalance challenges in real-world applications. Additionally, we introduce a Python package that facilitates the integration of class-balanced loss functions into GBDT workflows, making these advanced techniques accessible to a wider audience.

Machine Learning
MT-GBM: A Multi-Task Gradient Boosting Machine with Shared Decision Trees

ZhenZhe Ying,Zhuoer Xu,Zhifeng Li,Weiqiang Wang,Changhua Meng

DOI: https://doi.org/10.48550/arXiv.2201.06239

2022-01-24

Abstract:Despite the success of deep learning in computer vision and natural language processing, Gradient Boosted Decision Tree (GBDT) is yet one of the most powerful tools for applications with tabular data such as e-commerce and FinTech. However, applying GBDT to multi-task learning is still a challenge. Unlike deep models that can jointly learn a shared latent representation across multiple tasks, GBDT can hardly learn a shared tree structure. In this paper, we propose Multi-task Gradient Boosting Machine (MT-GBM), a GBDT-based method for multi-task learning. The MT-GBM can find the shared tree structures and split branches according to multi-task losses. First, it assigns multiple outputs to each leaf node. Next, it computes the gradient corresponding to each output (task). Then, we also propose an algorithm to combine the gradients of all tasks and update the tree. Finally, we apply MT-GBM to LightGBM. Experiments show that our MT-GBM improves the performance of the main task significantly, which means the proposed MT-GBM is efficient and effective.

Machine Learning
DeepGBM

Guolin Ke,Zhenhui Xu,Jia Zhang,Jiang Bian,Tie‐Yan Liu

DOI: https://doi.org/10.1145/3292500.3330858

2019-01-01

Abstract:Online prediction has become one of the most essential tasks in many real-world applications. Two main characteristics of typical online prediction tasks include tabular input space and online data generation. Specifically, tabular input space indicates the existence of both sparse categorical features and dense numerical ones, while online data generation implies continuous task-generated data with potentially dynamic distribution. Consequently, effective learning with tabular input space as well as fast adaption to online data generation become two vital challenges for obtaining the online prediction model. Although Gradient Boosting Decision Tree (GBDT) and Neural Network (NN) have been widely used in practice, either of them yields their own weaknesses. Particularly, GBDT can hardly be adapted to dynamic online data generation, and it tends to be ineffective when facing sparse categorical features; NN, on the other hand, is quite difficult to achieve satisfactory performance when facing dense numerical features. In this paper, we propose a new learning framework, DeepGBM, which integrates the advantages of the both NN and GBDT by using two corresponding NN components: (1) CatNN, focusing on handling sparse categorical features. (2) GBDT2NN, focusing on dense numerical features with distilled knowledge from GBDT. Powered by these two components, DeepGBM can leverage both categorical and numerical features while retaining the ability of efficient online update. Comprehensive experiments on a variety of publicly available datasets have demonstrated that DeepGBM can outperform other well-recognized baselines in various online prediction tasks.
Multi-grained and multi-layered gradient boosting decision tree for credit scoring

Wan’an Liu,Hong Fan,Min Xia

DOI: https://doi.org/10.1007/s10489-021-02715-6

IF: 5.3

2021-08-10

Applied Intelligence

Abstract:Credit scoring is an important process for banks and financial institutions to manage credit risk. Tree-based ensemble algorithms have made promising progress in credit scoring. However, tree-based ensemble algorithms lack representation learning, making them cannot well express the potential distribution of loan data. In this study, we propose a multi-grained and multi-layered gradient boosting decision tree (GBDT) for credit scoring. Multi-layered GBDT considers the advantages of the explicit learning process of tree-based model and the representation learning ability to discriminate good/bad applicants; multi-grained scanning augments original credit features while enhancing the representation learning ability of multi-layered GBDT. The experimental results on 6 credit scoring datasets show that the hierarchical structure can effectively reduce the intra-class distance and increase the inter-class distance of the credit scoring dataset. In addition, Multi-grained feature augmentation effectively increases the diversity of prediction and further improves the performance of credit scoring, providing more precise credit scoring results.

computer science, artificial intelligence
GPU-acceleration for Large-scale Tree Boosting

Huan Zhang,Si Si,Cho-Jui Hsieh

DOI: https://doi.org/10.48550/arXiv.1706.08359

2017-06-26

Abstract:In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step in Gradient Boosted Decision Tree (GBDT) and random forests training. Previous GPU based tree building algorithms are based on parallel multi-scan or radix sort to find the exact tree split, and thus suffer from scalability and performance issues. We show that using a histogram based algorithm to approximately find the best split is more efficient and scalable on GPU. By identifying the difference between classical GPU-based image histogram construction and the feature histogram construction in decision tree training, we develop a fast feature histogram building kernel on GPU with carefully designed computational and memory access sequence to reduce atomic update conflict and maximize GPU utilization. Our algorithm can be used as a drop-in replacement for histogram construction in popular tree boosting systems to improve their scalability. As an example, to train GBDT on epsilon dataset, our method using a main-stream GPU is 7-8 times faster than histogram based algorithm on CPU in LightGBM and 25 times faster than the exact-split finding algorithm in XGBoost on a dual-socket 28-core Xeon server, while achieving similar prediction accuracy.

Machine Learning,Distributed, Parallel, and Cluster Computing

An experimental evaluation of large scale GBDT systems

An Experimental Evaluation of Large Scale GBDT Systems

DimBoost

Challenges and Opportunities of Building Fast GBDT Systems.

Gradient Boosting With Piece-Wise Linear Regression Trees

Identifying Transportation Modes Using Gradient Boosting Decision Tree

Benchmarking and Optimization of Gradient Boosting Decision Tree Algorithms

SecureBoost+: Large Scale and High-Performance Vertical Federated Gradient Boosting Decision Tree

Gradient Boosted Binary Histogram Ensemble for Large-scale Regression

TencentBoost: A Gradient Boosting Tree System with Parameter Server

Label Aggregation of Gradient Boosting Decision Trees

Poster: gbdt-rs: Fast and Trustworthy Gradient Boosting Decision Tree

HarpGBDT: Optimizing Gradient Boosting Decision Tree for Parallel Efficiency

Unbiased Gradient Boosting Decision Tree with Unbiased Feature Importance

Quantized Training of Gradient Boosting Decision Trees

GBDT-MO: Gradient Boosted Decision Trees for Multiple Outputs

Improving GBDT Performance on Imbalanced Datasets: An Empirical Study of Class-Balanced Loss Functions

MT-GBM: A Multi-Task Gradient Boosting Machine with Shared Decision Trees

DeepGBM

Multi-grained and multi-layered gradient boosting decision tree for credit scoring

GPU-acceleration for Large-scale Tree Boosting