Fast Generation of Big Random Binary Trees

William B. Langdon

DOI: https://doi.org/10.48550/arXiv.2001.04505

2020-01-14

Abstract:random_tree() is a linear time and space C++ implementation able to create trees of up to a billion nodes for genetic programming and genetic improvement experiments. A 3.60GHz CPU can generate more than 18 million random nodes for GP program trees per second.

Data Structures and Algorithms,Neural and Evolutionary Computing

What problem does this paper attempt to address?

Optimal Generation of Strictly Increasing Binary Trees and Beyond

Olivier Bodini,Francis Durand,Philippe Marchal

DOI: https://doi.org/10.4204/EPTCS.403.15

2024-06-24

Abstract:This article presents two novel algorithms for generating random increasing trees. The first algorithm efficiently generates strictly increasing binary trees using an ad hoc method. The second algorithm improves the recursive method for weighted strictly increasing unary-binary increasing trees, optimizing randomness usage.

Data Structures and Algorithms,Discrete Mathematics
(Random) Trees of Intermediate Uniform Growth

George Kontogeorgiou,Martin Winter

2023-11-09

Abstract:For every sufficiently well-behaved function $g:\mathbb{R}_{\ge 0}\rightarrow\mathbb{R}_{\ge 0}$ that grows at least linearly and at most exponentially we construct a tree $T$ of uniform volume growth $g$, that is, $$C_1\cdot g(r/4)\le |B_{T}(v,r)| \le C_2\cdot g(4r),\quad\text{for all $r\ge 0$ and $v\in V(T)$},$$ where $B_{T}(v,r)$ denotes the ball of radius $r$ centered at a vertex $v$. In particular, this yields examples of trees of uniform intermediate (i.e. super-polynomial and sub-exponential) volume growth. We use this construction to provide first examples of unimodular random rooted trees of uniform intermediate growth, answering a question by Itai Benjamini. We find a peculiar change in structural properties for these trees at growth $r^{\log\log r}$.

Combinatorics,Probability
Holonomic equations and efficient random generation of binary trees

Pierre Lescanne

2024-01-23

Abstract:Holonomic equations are recursive equations which allow computing efficiently numbers of combinatoric objects. R{é}my showed that the holonomic equation associated with binary trees yields an efficient linear random generator of binary trees. I extend this paradigm to Motzkin trees and Schr{ö}der trees and show that despite slight differences my algorithm that generates random Schr{ö}der trees has linear expected complexity and my algorithm that generates Motzkin trees is in O(n) expected complexity, only if we can implement a specific oracle with a O(1) complexity. For Motzkin trees, I propose a solution which works well for realistic values (up to size ten millions) and yields an efficient algorithm.

Data Structures and Algorithms,Computational Complexity
Slow-Growing Trees

Philippe Goulet Coulombe

DOI: https://doi.org/10.48550/arXiv.2103.01926

2021-07-15

Abstract:Random Forest's performance can be matched by a single slow-growing tree (SGT), which uses a learning rate to tame CART's greedy algorithm. SGT exploits the view that CART is an extreme case of an iterative weighted least square procedure. Moreover, a unifying view of Boosted Trees (BT) and Random Forests (RF) is presented. Greedy ML algorithms' outcomes can be improved using either "slow learning" or diversification. SGT applies the former to estimate a single deep tree, and Booging (bagging stochastic BT with a high learning rate) uses the latter with additive shallow trees. The performance of this tree ensemble quaternity (Booging, BT, SGT, RF) is assessed on simulated and real regression tasks.

Machine Learning,Econometrics,Applications
Random trees have height $O(\sqrt{n})$

Louigi Addario-Berry,Serte Donderwinkel

2022-01-28

Abstract:We obtain new non-asymptotic tail bounds for the height of uniformly random trees with a given degree sequence, simply generated trees and conditioned Bienaym\'e trees (the family trees of branching processes), in the process settling three conjectures of Janson (2012) and answering several other questions from the literature. Moreover, we define a partial ordering on degree sequences and show that it induces a stochastic ordering on the heights of uniformly random trees with given degree sequences. The latter result can also be used to show that sub-binary random trees are stochastically the tallest trees with a given number of vertices and leaves (and thus that random binary trees are the stochastically tallest random homeomorphically irreducible trees with a given number of vertices). Our proofs are based in part on the bijection between trees and sequences introduced by Foata and Fuchs (1970), which can be recast to provide a line-breaking construction of random trees with given vertex degrees as shown in Addario-Berry, Blanc-Renaudie, Donderwinkel, Maazoun and Martin (2023).

Probability,Combinatorics
GPU-acceleration for Large-scale Tree Boosting

Huan Zhang,Si Si,Cho-Jui Hsieh

DOI: https://doi.org/10.48550/arXiv.1706.08359

2017-06-26

Abstract:In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step in Gradient Boosted Decision Tree (GBDT) and random forests training. Previous GPU based tree building algorithms are based on parallel multi-scan or radix sort to find the exact tree split, and thus suffer from scalability and performance issues. We show that using a histogram based algorithm to approximately find the best split is more efficient and scalable on GPU. By identifying the difference between classical GPU-based image histogram construction and the feature histogram construction in decision tree training, we develop a fast feature histogram building kernel on GPU with carefully designed computational and memory access sequence to reduce atomic update conflict and maximize GPU utilization. Our algorithm can be used as a drop-in replacement for histogram construction in popular tree boosting systems to improve their scalability. As an example, to train GBDT on epsilon dataset, our method using a main-stream GPU is 7-8 times faster than histogram based algorithm on CPU in LightGBM and 25 times faster than the exact-split finding algorithm in XGBoost on a dual-socket 28-core Xeon server, while achieving similar prediction accuracy.

Machine Learning,Distributed, Parallel, and Cluster Computing
On the Subtrees of Random Binary Search Trees

SU CHUN,MIAO BAIQI,FENG QUNQIANG

DOI: https://doi.org/10.3969/j.issn.1001-4268.2006.03.011

2006-01-01

Abstract:The subtrees of vatious sizes and patterns in random binary search trees are investigated in this paper.The expectations and variances of their numbers are first derived from an essential recursive distributional equation.Applying the contraction method,we show both of their asymptotic distributions are normal.
Barnes-hut Treecode on GPU

Hu Jiang,Qianni Deng

DOI: https://doi.org/10.1109/pic.2010.5687868

2010-01-01

Abstract:General-purpose computation on graphics processing units (GPGPU) has become a popular field of study. Due to its high computing capacity and relatively low price, GPU has been an ideal processing unit for many scientific applications, among which is N-body simulation. According to the published papers, a simple O(N^2) algorithm of N-body simulation has achieved some enhancements, but tree-algorithm doesn't work well on GPU. This paper proposes a new implementation of tree-algorithm on GPU using CUDA, which has obtained more than 100X speedup when computing forces between bodies. This paper also rises up a new method to build tree in this algorithm, making the performance even better.
A Template for Implementing Fast Lock-free Trees Using HTM

Trevor Brown

DOI: https://doi.org/10.48550/arXiv.1708.04838

2017-08-16

Abstract:Algorithms that use hardware transactional memory (HTM) must provide a software-only fallback path to guarantee progress. The design of the fallback path can have a profound impact on performance. If the fallback path is allowed to run concurrently with hardware transactions, then hardware transactions must be instrumented, adding significant overhead. Otherwise, hardware transactions must wait for any processes on the fallback path, causing concurrency bottlenecks, or move to the fallback path. We introduce an approach that combines the best of both worlds. The key idea is to use three execution paths: an HTM fast path, an HTM middle path, and a software fallback path, such that the middle path can run concurrently with each of the other two. The fast path and fallback path do not run concurrently, so the fast path incurs no instrumentation overhead. Furthermore, fast path transactions can move to the middle path instead of waiting or moving to the software path. We demonstrate our approach by producing an accelerated version of the tree update template of Brown et al., which can be used to implement fast lock-free data structures based on down-trees. We used the accelerated template to implement two lock-free trees: a binary search tree (BST), and an (a,b)-tree (a generalization of a B-tree). Experiments show that, with 72 concurrent processes, our accelerated (a,b)-tree performs between 4.0x and 4.2x as many operations per second as an implementation obtained using the original tree update template.

Distributed, Parallel, and Cluster Computing
TreeGen -- a Monte Carlo generator for data frames

Agnieszka Niemczynowicz,Gabriela Białoskórska,Joanna Nieżurawska-Zając,Radosław A. Kycia

DOI: https://doi.org/10.48550/arXiv.2011.08922

2020-11-18

Abstract:The typical problem in Data Science is creating a structure that encodes the occurrence frequency of unique elements in rows and relations between different rows of a data frame. We present the probability tree abstract data structure, an extension of the decision tree, that facilitates more than two choices with assigned probabilities. Such a tree represents statistical relations between different rows of the data frame. The Probability Tree algorithmic structure is supplied with the Generator module that is a Monte Carlo generator that traverses through the tree. These two components are implemented in TreeGen Python package. The package can be used in increasing data multiplicity, compressing data preserving its statistical information, constructing hierarchical models, exploring data, and in feature extraction.

Machine Learning,Computation
Amoeba Monte Carlo algorithms for random trees with controlled branching activity: Efficient trial move generation and universal dynamics

Pieter H. W. van der Hoek,Angelo Rosa,and Ralf Everaers

DOI: https://doi.org/10.1103/physreve.110.045312

IF: 2.707

2024-10-29

Physical Review E

Abstract:Simulating ensembles of branched macromolecules with annealed branches and statistically-controlled branching weights has been a long-standing challenge. The authors present a new algorithm for simulating random trees, advancing the computational theory of randomly branched polymers. [Phys. Rev. E 110, 045312] Published Mon Oct 28, 2024

physics, fluids & plasmas, mathematical
Fast genetic algorithms

Benjamin Doerr,Huu Phuoc Le,Régis Makhmara,Ta Duy Nguyen

DOI: https://doi.org/10.1145/3071178.3071301

2017-07-01

Abstract:For genetic algorithms (GAs) using a bit-string representation of length n, the general recommendation is to take 1/n as mutation rate. In this work, we discuss whether this is justified for multi-modal functions. Taking jump functions and the (1+1) evolutionary algorithm (EA) as the simplest example, we observe that larger mutation rates give significantly better runtimes. For the Jumpm, n function, any mutation rate between 2/n and m/n leads to a speedup at least exponential in m compared to the standard choice. The asymptotically best runtime, obtained from using the mutation rate m/n and leading to a speed-up super-exponential in m, is very sensitive to small changes of the mutation rate. Any deviation by a small (1 ε) factor leads to a slow-down exponential in m. Consequently, any fixed mutation rate gives strongly sub-optimal results for most jump functions. Building on this observation, we propose to use a random mutation rate /n, where is chosen from a power-law distribution. We prove that the (1 + 1) EA with this heavy-tailed mutation rate optimizes any Jumpm, n function in a time that is only a small polynomial (in m) factor above the one stemming from the optimal rate for this m. Our heavy-tailed mutation operator yields similar speed-ups (over the best known performance guarantees) for the vertex cover problem in bipartite graphs and the matching problem in general graphs. Following the example of fast simulated annealing, fast evolution strategies, and fast evolutionary programming, we propose to call genetic algorithms using a heavy-tailed mutation operator fast genetic algorithms.
Poster: gbdt-rs: Fast and Trustworthy Gradient Boosting Decision Tree

Tianyi Li,Tongxin Li,Yu Ding,Yulong Zhang,Tao Wei,Xinhui Han

2019-01-01

Abstract:For data analysis services, the capability of preserving user privacy has become an eagerly demanded trait. However, most existing approaches are either unsound to close some major information leakage chanels or prohibitively expensive for practical deployment. We propose gbdt-rs , a secure and fast implementation of the gradient boosting decision tree algorithm, which is widely used in data mining and machine learning tasks. gbdt-rs is designed to fit traditional computing platforms as well as hardware-enforced trusted environments, i.e., Intel SGX secure enclaves. gbdt-rs is coded in the Rust programming language to exterminate memory errors, while its performance is barely sacrificed due to sophisticatedly optimized memory access patterns. Our experiments show that gbdt-rs can provide up to 10x speedup on inference tasks compared to XGBoost, a reputed C++ implementation of the same algorithm. In addition, gbdt-rs is highly auditable due to its relatively small code base.
Enabling Rapid Development of Parallel Tree Search Applications

Jeffrey P. Gardner,Andrew Connolly,Cameron McBride

DOI: https://doi.org/10.48550/arXiv.0709.1967

2007-09-13

Abstract:Virtual observatories will give astronomers easy access to an unprecedented amount of data. Extracting scientific knowledge from these data will increasingly demand both efficient algorithms as well as the power of parallel computers. Nearly all efficient analyses of large astronomical datasets use trees as their fundamental data structure. Writing efficient tree-based techniques, a task that is time-consuming even on single-processor computers, is exceedingly cumbersome on massively parallel platforms (MPPs). Most applications that run on MPPs are simulation codes, since the expense of developing them is offset by the fact that they will be used for many years by many researchers. In contrast, data analysis codes change far more rapidly, are often unique to individual researchers, and therefore accommodate little reuse. Consequently, the economics of the current high-performance computing development paradigm for MPPs does not favor data analysis applications. We have therefore built a library, called Ntropy, that provides a flexible, extensible, and easy-to-use way of developing tree-based data analysis algorithms for both serial and parallel platforms. Our experience has shown that not only does our library save development time, it can also deliver excellent serial performance and parallel scalability. Furthermore, Ntropy makes it easy for an astronomer with little or no parallel programming experience to quickly scale their application to a distributed multiprocessor environment. By minimizing development time for efficient and scalable data analysis, we enable wide-scale knowledge discovery on massive datasets.

Astrophysics
Efficient phylogenetic tree inference for massive taxonomic datasets: harnessing the power of a server to analyze 1 million taxa

César Piñeiro,Juan C Pichel

DOI: https://doi.org/10.1093/gigascience/giae055

IF: 7.658

2024-01-02

GigaScience

Abstract:Background: Phylogenies play a crucial role in biological research. Unfortunately, the search for the optimal phylogenetic tree incurs significant computational costs, and most of the existing state-of-the-art tools cannot deal with extremely large datasets in reasonable times. Results: In this work, we introduce the new VeryFastTree code (version 4.0), which is able to construct a tree on 1 server using single-precision arithmetic from a massive 1 million alignment dataset in only 36 hours, which is 3 times and 3.2 times faster than its previous version and FastTree-2, respectively. This new version further boosts performance by parallelizing all tree traversal operations during the tree construction process, including subtree pruning and regrafting moves. Additionally, it introduces significant new features such as support for new and compressed file formats, enhanced compatibility across a broader range of operating systems, and the integration of disk computing functionality. The latter feature is particularly advantageous for users without access to high-end servers, as it allows them to manage very large datasets, albeit with an increase in computing time. Conclusions: Experimental results establish VeryFastTree as the fastest tool in the state-of-the-art for maximum likelihood phylogeny estimation. It is publicly available at https://github.com/citiususc/veryfasttree. In addition, VeryFastTree is included as a package in Bioconda, MacPorts, and all Debian-based Linux distributions.
Compound Binary Search Tree and Algorithms

Yong Tan

DOI: https://doi.org/10.48550/arXiv.1810.02270

2018-10-04

Abstract:The Binary Search Tree (BST) is average in computer science which supports a compact data structure in memory and oneself even conducts a row of quick algorithms, by which people often apply it in dynamical circumstance. Besides these edges, it is also with weakness on its own structure specially with poor performance at worst case. In this paper, we will develop this data structure into a synthesis to show a series of novel features residing in. Of that, there are new methods invented for raising the performance and efficiency nevertheless some existing ones in logarithm or linear time.

Data Structures and Algorithms,Databases
Burning Random Trees

Luc Devroye,Austin Eide,Pawel Pralat

2024-04-02

Abstract:Let $\mathcal{T}$ be a Galton-Watson tree with a given offspring distribution $\xi$, where $\xi$ is a $Z_{\geq 0}$-valued random variable with $E[\xi] = 1$ and $0 < \sigma^{2}:=Var[\xi] < \infty$. For $n \geq 1$, let $T_{n}$ be the tree $\mathcal{T}$ conditioned to have $n$ vertices. In this paper we investigate $b(T_n)$, the burning number of $T_n$. Our main result shows that asymptotically almost surely $b(T_n)$ is of the order of $n^{1/3}$.

Combinatorics,Probability
QuickTree: A Fast Hardware BVH Construction Engine

Yin Su,Hui Guo,Run Yan,Yong Wang,Yongwen Wang,Nong Xiao,Gang Chen,Weihua Zhang,Libo Huang

DOI: https://doi.org/10.1145/3649153.3649292

2024-01-01

Abstract:Ray tracing has emerged as a powerful technique for generating visually stunning and realistic images compared to rasterization. With the continuous advancements in computer hardware, modern GPUs have integrated specialized ray tracing acceleration units to enhance rendering capabilities further. However, achieving realtime ray tracing presents a challenge in dynamic scenes, where spatial data structures used for accelerated rendering must be reconstructed or updated when there are changes in the scene primitives. This paper introduces QuickTree, a novel Bounding Volume Hierarchy (BVH) construction engine based on the linear BVH (LBVH) optimization algorithm. QuickTree addresses the challenge of dynamic scenes support by employing a highly parallel and pipelined system design. This innovative approach ensures fast construction speed. QuickTree demonstrates significant performance improvements. Compared to the currently fastest MergeTree, it has increased construction speed by 10% and reduced area by 45% compared to RayCore, which has the smallest chip area.
Concurrent Binary Trees for Large-Scale Game Components

Anis Benyoub,Jonathan Dupuy

DOI: https://doi.org/10.1145/3675371

2024-07-02

Abstract:A concurrent binary tree (CBT) is a GPU-friendly data-structure suitable for the generation of bisection based terrain tessellations, i.e., adaptive triangulations over square domains. In this paper, we expand the benefits of this data-structure in two respects. First, we show how to bring bisection based tessellations to arbitrary polygon meshes rather than just squares. Our approach consists of mapping a triangular subdivision primitive, which we refer to as a bisector, to each halfedge of the input mesh. These bisectors can then be subdivided adaptively to produce conforming triangulations solely based on halfedge operators. Second, we alleviate a limitation that restricted the triangulations to low subdivision levels. We do so by using the CBT as a memory pool manager rather than an implicit encoding of the triangulation as done originally. By using a CBT in this way, we concurrently allocate and/or release bisectors during adaptive subdivision using shared GPU memory. We demonstrate the benefits of our improvements by rendering planetary scale geometry out of very coarse meshes. Performance-wise, our triangulation method evaluates in less than 0.2ms on console-level hardware.

Graphics

Fast Generation of Big Random Binary Trees

Optimal Generation of Strictly Increasing Binary Trees and Beyond

(Random) Trees of Intermediate Uniform Growth

Holonomic equations and efficient random generation of binary trees

Slow-Growing Trees

Random trees have height $O(\sqrt{n})$

GPU-acceleration for Large-scale Tree Boosting

On the Subtrees of Random Binary Search Trees

Barnes-hut Treecode on GPU

A Template for Implementing Fast Lock-free Trees Using HTM

TreeGen -- a Monte Carlo generator for data frames

Amoeba Monte Carlo algorithms for random trees with controlled branching activity: Efficient trial move generation and universal dynamics

Fast genetic algorithms

Poster: gbdt-rs: Fast and Trustworthy Gradient Boosting Decision Tree

Enabling Rapid Development of Parallel Tree Search Applications

Efficient phylogenetic tree inference for massive taxonomic datasets: harnessing the power of a server to analyze 1 million taxa

Compound Binary Search Tree and Algorithms

Burning Random Trees

QuickTree: A Fast Hardware BVH Construction Engine

Concurrent Binary Trees for Large-Scale Game Components