Abstract:Using empirical models to predict whether sections within pipes have defects can save inspection costs and, potentially, avoid oil spills. Optimal Classification Tree (OCT) formulations offer potentially desirable combinations of interpretability and prediction accuracy on unseen pipes. Approaches based on powerful state-of-the-art OCT formulations have enabled researchers to solve decision tree problems optimally instead of using traditional sub-optimal greedy approaches. Yet, the recently proposed formulations also have limitations. Some of the most recent formulations require a large number of decision variables and constraints leading to computational inefficiencies. Previous formulations have optimal solutions with undesirable or invalid tree structures which may depend on the particular software implementation. Additionally, some formulations always grow a full tree even when desirable parsimonious tree options are available. This article proposes the Modified Optimal Classification Tree (M-OCT) formulation with novel leaf-branch-interaction constraints, which could stabilize the previous formulation and reduce the chance of invalid tree structures when generating optimal trees. By incorporating the idea of binary encoding of thresholds from a previous article, we reduce the total number of binary variables. We then extend M-OCT to construct a novel formulation called Binary Node Penalty Optimal Classification Tree (BNP-OCT) with binary splits and node complexity constraints, which support efficiency in standard branch-and-cut solvers and prevents the overfitting issue when learning the optimal tree models. We compare the proposed methods with alternatives including standard formulations using 15 standard data sets. In addition, we use 750 test cases to compare the computational stability of pre-existing formulations to those involving the proposed leaf-branch constraints. We demonstrate that the proposed formulation offers advantages in accuracy, computational efficiency, and structural stability. We also describe how the proposed methods are able to achieve 94% classification accuracy on balanced test sets for unseen pipes.

Supplementary Material for Learning Optimal Tree Models under Beam Search

Learning Optimal Tree Models under Beam Search.

Learning Deep Tree-based Retriever for Efficient Recommendation: Theory and Method

Probabilistic Label Tree for Streaming Multi-Label Learning

BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solving

PLD-Tree: Persistent Laplacian Decision Tree for Protein-Protein Binding Free Energy Prediction

Optimal estimation of Gaussian (poly)trees

Gradient Boosting With Piece-Wise Linear Regression Trees

Learning Linear Gaussian Polytree Models with Interventions

Hybrid Bayesian Estimation Tree Learning with Discrete and Fuzzy Labels

Uncertainty-Guided Optimization on Large Language Model Search Trees

Computational Aspects of Optional Pólya Tree

Revealing the Unwritten: Visual Investigation of Beam Search Trees to Address Language Model Prompting Challenges

Optimal classification trees with leaf-branch and binary constraints

Are you going to the party: depends, who else is coming? [Learning hidden group dynamics via conditional latent tree models]

Policy Trees for Prediction: Interpretable and Adaptive Model Selection for Machine Learning

The research on analysis and improvement of logistic model tree

Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance

Mixed integer linear optimization formulations for learning optimal binary classification trees

Easy-First POS Tagging and Dependency Parsing with Beam Search.

Analyze Additive and Interaction Effects via Collaborative Trees