Abstract:Interpretable machine learning has emerged as central in leveraging artificial intelligence within high-stakes domains such as healthcare, where understanding the rationale behind model predictions is as critical as achieving high predictive accuracy. In this context, feature selection assumes a pivotal role in enhancing model interpretability by identifying the most important input features in black-box models. While random forests are frequently used in biomedicine for their remarkable performance on tabular datasets, the accuracy gained from aggregating decision trees comes at the expense of interpretability. Consequently, feature selection for enhancing interpretability in random forests has been extensively explored in supervised settings. However, its investigation in the unsupervised regime remains notably limited. To address this gap, the study introduces novel methods to construct feature graphs from unsupervised random forests and feature selection strategies to derive effective feature combinations from these graphs. Feature graphs are constructed for the entire dataset as well as individual clusters leveraging the parent-child node splits within the trees, such that feature centrality captures their relevance to the clustering task, while edge weights reflect the discriminating power of feature pairs. Graph-based feature selection methods are extensively evaluated on synthetic and benchmark datasets both in terms of their ability to reduce dimensionality while improving clustering performance, as well as to enhance model interpretability. An application on omics data for disease subtyping identifies the top features for each cluster, showcasing the potential of the proposed approach to enhance interpretability in clustering analyses and its utility in a real-world biomedical application.

Decision Predicate Graphs: Enhancing Interpretability in Tree Ensembles

Inherently Interpretable Tree Ensemble Learning

Interactive Decision Tree Creation and Enhancement with Complete Visualization for Explainable Modeling

Tree in Tree: from Decision Trees to Decision Graphs

Unboxing Tree Ensembles for interpretability: a hierarchical visualization tool and a multivariate optimal re-built tree

A generalized decision tree ensemble based on the NeuralNetworks architecture: Distributed Gradient Boosting Forest (DGBF)

TREE-G: Decision Trees Contesting Graph Neural Networks

Feature graphs for interpretable unsupervised tree ensembles: centrality, interaction, and application in disease subtyping

Optimization of Tree Ensembles

RuleExplorer: A Scalable Matrix Visualization for Understanding Tree Ensemble Classifiers

SONG: Self-Organizing Neural Graphs

Ensemble Predicate Decoding for Unbiased Scene Graph Generation

Learn Decision Trees with Deep Visual Primitives

Born-Again Tree Ensembles

Topological Interpretability for Deep-Learning

Extracting Optimal Explanations for Ensemble Trees Via Automated Reasoning

Interpretable Decision Tree Search as a Markov Decision Process

Representing and extending ensembles of parsimonious evolutionary histories with a directed acyclic graph

Interpretability Diversity for Decision-Tree-Initialized Dendritic Neuron Model Ensemble

GPTree: Towards Explainable Decision-Making via LLM-powered Decision Trees

t-PINE: tensor-based predictable and interpretable node embeddings