Abstract:Tsetlin Machines (TMs) have emerged as a compelling alternative to conventional deep learning methods, offering notable advantages such as smaller memory footprint, faster inference, fault-tolerant properties, and interpretability. Although various adaptations of TMs have expanded their applicability across diverse domains, a fundamental gap remains in understanding how TMs quantify uncertainty in their predictions. In response, this paper introduces the Probabilistic Tsetlin Machine (PTM) framework, aimed at providing a robust, reliable, and interpretable approach for uncertainty quantification. Unlike the original TM, the PTM learns the probability of staying on each state of each Tsetlin Automaton (TA) across all clauses. These probabilities are updated using the feedback tables that are part of the TM framework: Type I and Type II feedback. During inference, TAs decide their actions by sampling states based on learned probability distributions, akin to Bayesian neural networks when generating weight values. In our experimental analysis, we first illustrate the spread of the probabilities across TA states for the noisy-XOR dataset. Then we evaluate the PTM alongside benchmark models using both simulated and real-world datasets. The experiments on the simulated dataset reveal the PTM's effectiveness in uncertainty quantification, particularly in delineating decision boundaries and identifying regions of high uncertainty. Moreover, when applied to multiclass classification tasks using the Iris dataset, the PTM demonstrates competitive performance in terms of predictive entropy and expected calibration error, showcasing its potential as a reliable tool for uncertainty estimation. Our findings underscore the importance of selecting appropriate models for accurate uncertainty quantification in predictive tasks, with the PTM offering a particularly interpretable and effective solution.

Representing uncertainty and imprecision in machine learning: A survey on belief functions

An Improved Belief Entropy and Its Application in Decision-Making

Application of belief functions to medical image segmentation: A review

A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice

Uncertainty as a Fairness Measure

A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning

Measuring Classification Decision Certainty and Doubt

Uncertainty-Based Rejection in Machine Learning: Implications for Model Development and Interpretability

Information Fusion on Belief Networks

Divergence Measure of Belief Function and Its Application in Data Fusion

Introducing an Improved Information-Theoretic Measure of Predictive Uncertainty

Uncertainty-based Fairness Measures

The Probabilistic Tsetlin Machine: A Novel Approach to Uncertainty Quantification

A New Correlation Measure for Belief Functions and Their Application in Data Fusion

A Survey on Uncertainty Quantification Methods for Deep Learning

The basic principles of uncertain information fusion. An organised review of merging rules in different representation frameworks

Uncertainties of Latent Representations in Computer Vision

Uncertainty-Aware Bayes' Rule and Its Applications

Uncertainty Quantification in Logistic Regression using Random Fuzzy Sets and Belief Functions

Uncertainty theory as a basis for belief reliability

Towards Uncertainty in Decision: A Survey on Recent Advances and Challenges in Bayesian Reinforcement Learning