Abstract:Understanding recurrent networks through rule extraction has a long history. This has taken on new interests due to the need for interpreting or verifying neural networks. One basic form for representing stateful rules is deterministic finite automata (DFA). Previous research shows that extracting DFAs from trained second-order recurrent networks is not only possible but also relatively stable. Recently, several new types of recurrent networks with more complicated architectures have been introduced. These handle challenging learning tasks usually involving sequential data. However, it remains an open problem whether DFAs can be adequately extracted from these models. Specifically, it is not clear how DFA extraction will be affected when applied to different recurrent networks trained on data sets with different levels of complexity. Here, we investigate DFA extraction on several widely adopted recurrent networks that are trained to learn a set of seven regular Tomita grammars. We first formally analyze the complexity of Tomita grammars and categorize these grammars according to that complexity. Then we empirically evaluate different recurrent networks for their performance of DFA extraction on all Tomita grammars. Our experiments show that for most recurrent networks, their extraction performance decreases as the complexity of the underlying grammar increases. On grammars of lower complexity, most recurrent networks obtain desirable extraction performance. As for grammars with the highest level of complexity, while several complicated models fail with only certain recurrent networks having satisfactory extraction performance.

State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictions

Towards Interpreting Recurrent Neural Networks Through Probabilistic Abstraction

Residual Recurrent Neural Networks for Learning Sequential Representations.

Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks.

Weighted Automata Extraction and Explanation of Recurrent Neural Networks for Natural Language Tasks

A Comparative Study of Rule Extraction for Recurrent Neural Networks

DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction

Extracting Weighted Finite Automata from Recurrent Neural Networks for Natural Languages

Understanding Hidden Memories of Recurrent Neural Networks

Learning minimal automata with recurrent neural networks

Recurrent Neural Network Regularization

Cold-start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks

Evaluating Recurrent Neural Network Explanations

DRRNets: Dynamic Recurrent Routing Via Low-Rank Regularization in Recurrent Neural Networks.

Predictive-State Decoders: Encoding the Future into Recurrent Networks

Subregular Complexity and Deep Learning

Alleviate Exposure Bias in Sequence Prediction \\ with Recurrent Neural Networks

Why are state-space models more expressive than $n$-gram models?

Recurrently Controlled Recurrent Networks

Advancing Regular Language Reasoning in Linear Recurrent Neural Networks

Interpreting recurrent neural networks behaviour via excitable network attractors