Abstract:Tree-based models for probability distributions are usually specified using a predetermined, data-independent collection of candidate recursive partitions of the sample space. To characterize an unknown target density in detail over the entire sample space, candidate partitions must have the capacity to expand deeply into all areas of the sample space with potential non-zero sampling probability. Such an expansive system of partitions often incurs prohibitive computational costs and makes inference prone to overfitting, especially in regions with little probability mass. Existing models typically make a compromise and rely on relatively shallow trees. This hampers one of the most desirable features of trees, their ability to characterize local features, and results in reduced statistical efficiency. Traditional wisdom suggests that this compromise is inevitable to ensure coherent likelihood-based reasoning, as a data-dependent partition system that allows deeper expansion only in regions with more observations would induce double dipping of the data and thus lead to inconsistent inference. We propose a simple strategy to restore coherency while allowing the candidate partitions to be data-dependent, using Cox's partial likelihood. This strategy parametrizes the tree-based sampling model according to the allocation of probability mass based on the observed data, and yet under appropriate specification, the resulting inference remains valid. Our partial likelihood approach is broadly applicable to existing likelihood-based methods and in particular to Bayesian inference on tree-based models. We give examples in density estimation in which the partial likelihood is endowed with existing priors on tree-based models and compare with the standard, full-likelihood approach. The results show substantial gains in estimation accuracy and computational efficiency from using the partial likelihood.

Generalizing Tree Probability Estimation Via Bayesian Networks

Improving Tree Probability Estimation with Stochastic Optimization and Variance Reduction

Learning Bayesian Networks by Constrained Bayesian Estimation

A partial likelihood approach to tree-based density modeling and its application in Bayesian inference

Hybrid Bayesian Estimation Tree Learning with Discrete and Fuzzy Labels

Learning Bayesian Networks Using the Constrained Maximum a Posteriori Probability Method

A Comparative Analysis of Methods for Probability Estimation Tree

A Factor Tree Inference Algorithm for Bayesian Networks and Its Application

Bayesian Out-Trees

Improving Tree Augmented Naive Bayes for Class Probability Estimation

On Testing Whether an Embedded Bayesian Network Represents a Probability Model

Probability Distribution on Rooted Trees

Accurate Bayesian phylogenetic point estimation using a tree distribution parameterized by clade probabilities

Probability Distribution on Full Rooted Trees

Bayesian learning of forest and tree graphical models

Scaling Up the Accuracy of Bayesian Network Classifiers by M-Estimate

Bayesian Decision Trees via Tractable Priors and Probabilistic Context-Free Grammars

A Computational Method for Bounding the Probability of Reconstruction on Trees

Application of Bayesian Networks in Dynamic Event Tree Analysis

Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions

Decision Tree with Better Class Probability Estimation.