Abstract:Using empirical models to predict whether sections within pipes have defects can save inspection costs and, potentially, avoid oil spills. Optimal Classification Tree (OCT) formulations offer potentially desirable combinations of interpretability and prediction accuracy on unseen pipes. Approaches based on powerful state-of-the-art OCT formulations have enabled researchers to solve decision tree problems optimally instead of using traditional sub-optimal greedy approaches. Yet, the recently proposed formulations also have limitations. Some of the most recent formulations require a large number of decision variables and constraints leading to computational inefficiencies. Previous formulations have optimal solutions with undesirable or invalid tree structures which may depend on the particular software implementation. Additionally, some formulations always grow a full tree even when desirable parsimonious tree options are available. This article proposes the Modified Optimal Classification Tree (M-OCT) formulation with novel leaf-branch-interaction constraints, which could stabilize the previous formulation and reduce the chance of invalid tree structures when generating optimal trees. By incorporating the idea of binary encoding of thresholds from a previous article, we reduce the total number of binary variables. We then extend M-OCT to construct a novel formulation called Binary Node Penalty Optimal Classification Tree (BNP-OCT) with binary splits and node complexity constraints, which support efficiency in standard branch-and-cut solvers and prevents the overfitting issue when learning the optimal tree models. We compare the proposed methods with alternatives including standard formulations using 15 standard data sets. In addition, we use 750 test cases to compare the computational stability of pre-existing formulations to those involving the proposed leaf-branch constraints. We demonstrate that the proposed formulation offers advantages in accuracy, computational efficiency, and structural stability. We also describe how the proposed methods are able to achieve 94% classification accuracy on balanced test sets for unseen pipes.

Nonlinear Optimal Classification Trees

A nonlinear predictive control algorithm based on fuzzy online modeling and discrete optimization

Mixed-Integer Linear Optimization for Semi-Supervised Optimal Classification Trees

Strong Optimal Classification Trees

Optimal Mixed Integer Linear Optimization Trained Multivariate Classification Trees

Mixed integer linear optimization formulations for learning optimal binary classification trees

Rolling Lookahead Learning for Optimal Classification Trees

Sparsity in Optimal Randomized Classification Trees

Optimal classification trees with leaf-branch and binary constraints

Optimal Sparse Regression Trees

Learning Optimal Classification Trees Robust to Distribution Shifts

Efficient non-greedy optimization of decision trees

BooleanOCT: Optimal Classification Trees based on multivariate Boolean Rules

Structural Optimization Makes Graph Classification Simpler and Better

Optimal Sparse Decision Trees

Quant-BnB: A Scalable Branch-and-Bound Method for Optimal Decision Trees with Continuous Features

An improved column-generation-based matheuristic for learning classification trees

A Mathematical Programming Approach to Optimal Classification Forests

The Max-Cut Decision Tree: Improving on the Accuracy and Running Time of Decision Trees

Algorithm for Constructing Fast Classification Trees

Robust Decision Trees Against Adversarial Examples