Abstract:Rule set learning has long been studied and has recently been frequently revisited due to the need for interpretable models. Still, existing methods have several shortcomings: 1) most recent methods require a binary feature matrix as input, while learning rules directly from numeric variables is understudied; 2) existing methods impose orders among rules, either explicitly or implicitly, which harms interpretability; and 3) currently no method exists for learning probabilistic rule sets for multi-class target variables (there is only one for probabilistic rule lists). We propose TURS, for Truly Unordered Rule Sets, which addresses these shortcomings. We first formalize the problem of learning truly unordered rule sets. To resolve conflicts caused by overlapping rules, i.e., instances covered by multiple rules, we propose a novel approach that exploits the probabilistic properties of our rule sets. We next develop a two-phase heuristic algorithm that learns rule sets by carefully growing rules. An important innovation is that we use a surrogate score to take the global potential of the rule set into account when learning a local rule. Finally, we empirically demonstrate that, compared to non-probabilistic and (explicitly or implicitly) ordered state-of-the-art methods, our method learns rule sets that not only have better interpretability but also better predictive performance.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve several key problems in existing rule - set learning methods, as follows: 1. **Deficiencies in learning rules directly from numerical variables**: - Most existing methods require a binary feature matrix as input, and relatively few studies learn rules directly from numerical variables. 2. **The influence of the order between rules on interpretability**: - Existing methods usually impose an order, either explicitly or implicitly, between rules, which undermines the interpretability of the model. For example, when an instance satisfies multiple rules, existing methods usually rank these rules according to certain criteria and select the rule with the highest rank, thus introducing implicit order. 3. **The lack of learning probability rule sets in multi - class classification**: - Currently, no method can learn probability rule sets for multi - class target variables (only one method can learn a probability rule list). To solve these problems, the authors propose **Truly Unordered Rule Sets (Turs)**, that is, truly unordered probability rule sets. The main contributions of this method include: - **Formally defining truly unordered probability rule sets**: Solving the prediction conflict problem caused by rule overlap. - **Proposing a novel method to handle overlapping rules**: Using the probabilistic nature of the rule set to solve the conflicts caused by overlapping rules. - **Developing a two - stage heuristic algorithm**: Learning the rule set by carefully expanding the rules while considering the global potential. - **Introducing a surrogate scoring mechanism**: Considering the global potential of the entire rule set when learning local rules. - **Experimental verification**: Compared with non - probabilistic and (explicitly or implicitly) ordered state - of - the - art methods, the rule sets learned by Turs not only have better interpretability but also better prediction performance. In summary, this paper is committed to constructing a more interpretable and efficient multi - class classification model, especially in sensitive practical application scenarios (such as healthcare), where analysts need to be able to provide clear and reliable explanations.

Truly Unordered Probabilistic Rule Sets for Multi-class Classification

Probabilistic Truly Unordered Rule Sets

SIRUS: Stable and Interpretable RUle Set for classification

Probabilistic Safety Regions Via Finite Families of Scalable Classifiers

Multiple-category Classification with Decision-Theoretic Rough Sets

An Interpretable Rule Creation Method for Black-Box Models based on Surrogate Trees -- SRules

A Unified Approach to Extract Interpretable Rules from Tree Ensembles via Integer Programming

Rule Generation for Classification: Scalability, Interpretability, and Fairness

Interpretable Random Forests via Rule Extraction

Neural Network-Based Rule Models With Truth Tables

Better Short than Greedy: Interpretable Models through Optimal Rule Boosting

Neuro-Symbolic Rule Lists

Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets

LIBRE: Learning Interpretable Boolean Rule Ensembles

Bayesian Rule Set: A Quantitative Alternative to Qualitative Comparative Analysis

Learning Fuzzy β-Certain and β-Possible rules from incomplete quantitative data by rough sets

A Scalable Two Stage Approach to Computing Optimal Decision Sets

A rule-general abductive learning by rough sets

CauseRuDi: Explaining Behavior Sequence Models by Causal Statistics Generation and Rule Distillation

Learning Fair Rule Lists

Learning Certifiably Optimal Rule Lists for Categorical Data