Abstract:Machine learning has become omnipresent with applications in various safety-critical domains such as medical, law, and transportation. In these domains, high-stake decisions provided by machine learning necessitate researchers to design interpretable models, where the prediction is understandable to a human. In interpretable machine learning, rule-based classifiers are particularly effective in representing the decision boundary through a set of rules comprising input features. The interpretability of rule-based classifiers is in general related to the size of the rules, where smaller rules are considered more interpretable. To learn such a classifier, the brute-force direct approach is to consider an optimization problem that tries to learn the smallest classification rule that has close to maximum accuracy. This optimization problem is computationally intractable due to its combinatorial nature and thus, the problem is not scalable in large datasets. To this end, in this paper we study the triangular relationship among the accuracy, interpretability, and scalability of learning rule-based classifiers. The contribution of this paper is an interpretable learning framework IMLI, that is based on maximum satisfiability (MaxSAT) for synthesizing classification rules expressible in proposition logic. Despite the progress of MaxSAT solving in the last decade, the straightforward MaxSAT-based solution cannot scale. Therefore, we incorporate an efficient incremental learning technique inside the MaxSAT formulation by integrating mini-batch learning and iterative rule-learning. In our experiments, IMLI achieves the best balance among prediction accuracy, interpretability, and scalability. As an application, we deploy IMLI in learning popular interpretable classifiers such as decision lists and decision sets.

Interpretable Decision Trees Through MaxSAT

Learning Optimal Decision Trees with SAT

Interpretable Decision Tree Search as a Markov Decision Process

Optimal Interpretability-Performance Trade-off of Classification Trees with Black-Box Reinforcement Learning

A Scalable Two Stage Approach to Computing Optimal Decision Sets

Witty: An Efficient Solver for Computing Minimum-Size Decision Trees

Learning accurate and interpretable decision trees

Optimal Decision Lists using SAT

A Uniform Language to Explain Decision Trees

Efficient Learning of Interpretable Classification Rules

Rectified Decision Trees: Exploring the Landscape of Interpretable and Effective Machine Learning

The Max-Cut Decision Tree: Improving on the Accuracy and Running Time of Decision Trees

Limits of Actor-Critic Algorithms for Decision Tree Policies Learning in IBMDPs

An Incremental MaxSAT-based Model to Learn Interpretable and Balanced Classification Rules

Rectified Decision Trees: Towards Interpretability, Compression and Empirical Soundness

On the Safety of Interpretable Machine Learning: A Maximum Deviation Approach

Making Decision Trees Feasible in Ultrahigh Feature and Label Dimensions.

Interactive Decision Tree Creation and Enhancement with Complete Visualization for Explainable Modeling

Deep Trees for (Un)structured Data: Tractability, Performance, and Interpretability

dtControl 2.0: Explainable Strategy Representation via Decision Tree Learning Steered by Experts

Optimal Decision Tree Policies for Markov Decision Processes