Abstract:Neural networks are becoming a popular tool for solving many real-world problems such as object recognition and machine translation, thanks to its exceptional performance as an end-to-end solution. However, neural networks are complex black-box models, which hinders humans from interpreting and consequently trusting them in making critical decisions. Towards interpreting neural networks, several approaches have been proposed to extract simple deterministic models from neural networks. The results are not encouraging (e.g., low accuracy and limited scalability), fundamentally due to the limited expressiveness of such simple models. In this work, we propose an approach to extract probabilistic automata for interpreting an important class of neural networks, i.e., recurrent neural networks. Our work distinguishes itself from existing approaches in two important ways. One is that probability is used to compensate for the loss of expressiveness. This is inspired by the observation that human reasoning is often `probabilistic'. The other is that we adaptively identify the right level of abstraction so that a simple model is extracted in a request-specific way. We conduct experiments on several real-world datasets using state-of-the-art architectures including GRU and LSTM. The result shows that our approach significantly improves existing approaches in terms of accuracy or scalability. Lastly, we demonstrate the usefulness of the extracted models through detecting adversarial texts.

Analyzing and Exploiting NARX Recurrent Neural Networks for Long-Term Dependencies

Residual Recurrent Neural Networks for Learning Sequential Representations.

Towards Interpreting Recurrent Neural Networks Through Probabilistic Abstraction

Long-term time series prediction with the NARX network: An empirical evaluation

Residual Memory Networks: Feed-forward approach to learn long temporal dependencies

Learning Longer Memory in Recurrent Neural Networks

Resurrecting Recurrent Neural Networks for Long Sequences

Non-local Recurrent Neural Memory for Supervised Sequence Modeling

Optimizing and Contrasting Recurrent Neural Network Architectures

Time series prediction based on NARX neural networks: An advanced approach

A delay damage model selection algorithm for NARX neural networks

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics

Relational recurrent neural networks

Long short-term memory based on a reward/punishment strategy for recurrent neural networks

AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks

Learning Long-Term Dependencies in Irregularly-Sampled Time Series

XNAP: Making LSTM-based Next Activity Predictions Explainable by Using LRP

ARLIF: A Flexible and Efficient Recurrent Neuronal Model for Sequential Tasks.

On extended long short-term memory and dependent bidirectional recurrent neural network

Recurrent Neural Machine Translation.

An Intrinsic Nearest Neighbor Analysis of Neural Machine Translation Architectures