Abstract:Understanding recurrent networks through rule extraction has a long history. This has taken on new interests due to the need for interpreting or verifying neural networks. One basic form for representing stateful rules is deterministic finite automata (DFA). Previous research shows that extracting DFAs from trained second-order recurrent networks is not only possible but also relatively stable. Recently, several new types of recurrent networks with more complicated architectures have been introduced. These handle challenging learning tasks usually involving sequential data. However, it remains an open problem whether DFAs can be adequately extracted from these models. Specifically, it is not clear how DFA extraction will be affected when applied to different recurrent networks trained on data sets with different levels of complexity. Here, we investigate DFA extraction on several widely adopted recurrent networks that are trained to learn a set of seven regular Tomita grammars. We first formally analyze the complexity of Tomita grammars and categorize these grammars according to that complexity. Then we empirically evaluate different recurrent networks for their performance of DFA extraction on all Tomita grammars. Our experiments show that for most recurrent networks, their extraction performance decreases as the complexity of the underlying grammar increases. On grammars of lower complexity, most recurrent networks obtain desirable extraction performance. As for grammars with the highest level of complexity, while several complicated models fail with only certain recurrent networks having satisfactory extraction performance.

Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks

Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks.

Weighted Automata Extraction and Explanation of Recurrent Neural Networks for Natural Language Tasks

Extracting Weighted Finite Automata from Recurrent Neural Networks for Natural Languages

Towards Interpreting Recurrent Neural Networks Through Probabilistic Abstraction

Explaining Black Boxes on Sequential Data using Weighted Automata

Analyzing Recurrent Neural Network by Probabilistic Abstraction

Residual Recurrent Neural Networks for Learning Sequential Representations.

State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictions

A Comparative Study of Rule Extraction for Recurrent Neural Networks

A Decision-Making Method for Air Combat Maneuver Based on Hybrid Deep Learning Network

Learning Useful Representations of Recurrent Neural Network Weight Matrices

On Attribution of Recurrent Neural Network Predictions via Additive Decomposition

Multiple-Weight Recurrent Neural Networks.

Machine Learning on Sequential Data Using a Recurrent Weighted Average

Automata Extraction from Transformers

DeepCover: Advancing RNN test coverage and online error prediction using state machine extraction

WeNet: Weighted Networks for Recurrent Network Architecture Search

Learning minimal automata with recurrent neural networks

Simulating Weighted Automata over Sequences and Trees with Transformers