Abstract:One of the potential solutions for model interpretation is to train a surrogate model: a more transparent model that approximates the behavior of the model to be explained. Typically, classification rules or decision trees are used due to the intelligibility of their logic-based expressions. However, decision trees can grow too deep and rule sets can become too large to approximate a complex model. Unlike paths on a decision tree that must share ancestor nodes (conditions), rules are more flexible. However, the unstructured visual representation of rules makes it hard to make inferences across rules. To address these issues, we present a workflow that includes novel algorithmic and interactive solutions. First, we present Hierarchical Surrogate Rules (HSR), an algorithm that generates hierarchical rules based on user-defined parameters. We also contribute SuRE, a visual analytics (VA) system that integrates HSR and interactive surrogate rule visualizations. Particularly, we present a novel feature-aligned tree to overcome the shortcomings of existing rule visualizations. We evaluate the algorithm in terms of parameter sensitivity, time performance, and comparison with surrogate decision trees and find that it scales reasonably well and outperforms decision trees in many respects. We also evaluate the visualization and the VA system by a usability study with 24 volunteers and an observational study with 7 domain experts. Our investigation shows that the participants can use feature-aligned trees to perform non-trivial tasks with very high accuracy. We also discuss many interesting observations that can be useful for future research on designing effective rule-based VA systems.

Visualizing inference process of a rule engine

Learning the Dynamics of Visual Relational Reasoning Via Reinforced Path Routing

RLogic: Recursive Logical Rule Learning from Knowledge Graphs

Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets

Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning

RuleMatrix: Visualizing and Understanding Classifiers with Rules

Reasoning Makes Good Annotators : an Automatic Task-specific Rules Distilling Framework for Low-resource Relation Extraction

Visual Analysis of User-Driven Association Rule Mining

A distributed architecture for rule engine to deal with big data

The Rete Algorithm Improvement and Implementation

Fiper: a Visual-based Explanation Combining Rules and Feature Importance

RulE: Neural-Symbolic Knowledge Graph Reasoning with Rule Embedding

Visualizing the Evaluation of Functional Programs for Debugging

Learning differentiable logic programs for abstract visual reasoning

Learning First-Order Rules with Relational Path Contrast for Inductive Relation Reasoning

RulE: Knowledge Graph Reasoning with Rule Embedding

Declarative Demand-Driven Reverse Engineering

RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs

REVEAL-IT: REinforcement learning with Visibility of Evolving Agent poLicy for InTerpretability

Rule-based runtime verification revisited

Prolog Visualization System Using Logichart Diagrams