Abstract:Using finite-state machines to learn patterns, Tsetlin machines (TMs) have obtained competitive accuracy and learning speed across several benchmarks, with frugal memory- and energy footprint. A TM represents patterns as conjunctive clauses in propositional logic (AND-rules), each clause voting for or against a particular output. While efficient for single-output problems, one needs a separate TM per output for multi-output problems. Employing multiple TMs hinders pattern reuse because each TM then operates in a silo. In this paper, we introduce clause sharing, merging multiple TMs into a single one. Each clause is related to each output by using a weight. A positive weight makes the clause vote for output $1$, while a negative weight makes the clause vote for output $0$. The clauses thus coalesce to produce multiple outputs. The resulting coalesced Tsetlin Machine (CoTM) simultaneously learns both the weights and the composition of each clause by employing interacting Stochastic Searching on the Line (SSL) and Tsetlin Automata (TA) teams. Our empirical results on MNIST, Fashion-MNIST, and Kuzushiji-MNIST show that CoTM obtains significantly higher accuracy than TM on $50$- to $1$K-clause configurations, indicating an ability to repurpose clauses. E.g., accuracy goes from $71.99$% to $89.66$% on Fashion-MNIST when employing $50$ clauses per class (22 Kb memory). While TM and CoTM accuracy is similar when using more than $1$K clauses per class, CoTM reaches peak accuracy $3\times$ faster on MNIST with $8$K clauses. We further investigate robustness towards imbalanced training data. Our evaluations on imbalanced versions of IMDb- and CIFAR10 data show that CoTM is robust towards high degrees of class imbalance. Being able to share clauses, we believe CoTM will enable new TM application domains that involve multiple outputs, such as learning language models and auto-encoding.

Learning Moore Machines from Input-Output Traces

Output-decomposed Learning of Mealy Machines

Moore machines duality

Extracting Finite State Machines from Transformers

The semibicategory of Moore automata

Human in the Loop: Interactive Passive Automata Learning via Evidence-Driven State-Merging Algorithms

Grammatical Inference as a Satisfiability Modulo Theories Problem

Generalized Convergence Analysis of Tsetlin Machines: A Probabilistic Approach to Concept Learning

Learning Quantitative Automata Modulo Theories

A Passive Online Technique for Learning Hybrid Automata from Input/Output Traces

Learning state machines via efficient hashing of future traces

Complexity of Equivalence and Learning for Multiplicity Tree Automata

Learning Explainable and Better Performing Representations of POMDP Strategies

Model Representation and Cooperative Coevolution for Finite-State Machine Evolution.

Coalesced Multi-Output Tsetlin Machines with Clause Sharing

A Scheme-Driven Approach to Learning Programs from Input/Output Equations

The Tsetlin Machine -- A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic

$L^*LM$: Learning Automata from Examples using Natural Language Oracles

Automata Learning from Preference and Equivalence Queries

Active Learning of Mealy Machines with Timers

Small Test Suites for Active Automata Learning