Abstract:Counter machines have achieved a newfound relevance to the field of natural language processing (NLP): recent work suggests some strong-performing recurrent neural networks utilize their memory as counters. Thus, one potential way to understand the success of these networks is to revisit the theory of counter computation. Therefore, we study the abilities of real-time counter machines as formal grammars, focusing on formal properties that are relevant for NLP models. We first show that several variants of the counter machine converge to express the same class of formal languages. We also prove that counter languages are closed under complement, union, intersection, and many other common set operations. Next, we show that counter machines cannot evaluate boolean expressions, even though they can weakly validate their syntax. This has implications for the interpretability and evaluation of neural network systems: successfully matching syntactic patterns does not guarantee that counter memory accurately encodes compositional semantics. Finally, we consider whether counter languages are semilinear. This work makes general contributions to the theory of formal languages that are of potential interest for understanding recurrent neural networks.

On co-counter-fragments of automata

Counter Machines and Distributed Automata: A Story about Exchanging Space and Time

Three Lectures on Automatic Structures

Elementary fractal geometry. 4. Automata-generated topological spaces

Arbitrary-arity Tree Automata and QCTL

On Automata with Boundary

Towards the Notion of an Abstract Quantum Automaton

Conway's cosmological theorem and automata theory

Effective Characterizations of Simple Fragments of Temporal Logic Using Carton--Michel Automata

Learning Realtime One-Counter Automata

On Shuffling and Splitting Automata

In Memory of Oded Maler

Fractals from Regular Behaviours

Alternating Quantifiers in Uniform One-Dimensional Fragments with an Excursion into Three-Variable Logic

Expressiveness modulo Bisimilarity of Regular Expressions with Parallel Composition (Extended Abstract)

The Adjacent Fragment and Quine's Limits of Decision

Primitive Automata that are Synchronizing

On the Linguistic Capacity of Real-Time Counter Automata

Positive and monotone fragments of FO and LTL

Unambiguous and Co-Nondeterministic Computations of Finite Automata and Pushdown Automata Families and the Effects of Multiple Counters

Opacités Des Automates Finis