Abstract:The Transformer models have achieved unprecedented breakthroughs in text classification, and have become the foundation of most state-of-the-art NLP systems. The core function that drives the success is the attention mechanism, which provides the ability to dynamically focus on different parts of the input sequence when producing the predictions. Several previous works have investigated the usage of attention weights to explain the model predictions, because intuitively, attention weights reflect the importance of the input positions in the output. Specifically, the objective for explanation is to compute a relevance score for each input token, such that the key input words that are most important to the prediction can be identified. However, previous efforts produced mixed results. We find that the key reason why attention weights cannot be directly used as effective relevance indications is because they do not contain the directional information for relevance (i.e., whether the input tokens contribute towards or against the prediction). We then propose two novel explanation techniques, namely AGrad and RePAGrad, that produce directional relevance scores based on attention weights. To evaluate the explanation performance, we propose three properties that an effective explanation method should satisfy (i.e., faithfulness, resilience, and consistency), and design the corresponding test to quantify each property. Through extensive evaluations with Transformer models and pre-trained BERT models on multiple public text classification datasets, we show that AGrad and RePAGrad significantly outperform existing state-of-the-art explanation methods in faithfulness and consistency, at the cost of nominal degradation on resilience compared to attention weights. In addition, we reveal that elements of a model architecture can play an important role towards explainability.

A Novel Explainable Structure for Text Classification

SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers

Towards Explainable NLP: A Generative Explanation Framework for Text Classification

A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification

Generating Hierarchical Explanations on Text Classification Without Connecting Rules

Hierarchical Interpretation of Neural Text Classification

Self-Explaining Structures Improve NLP Models

Explainability of Text Processing and Retrieval Methods: A Critical Survey

Explaining the Predictions of Any Image Classifier via Decision Trees

From outputs to insights: a survey of rationalization approaches for explainable text classification

Learning to Annotate: Modularizing Data Augmentation for Text Classifiers with Natural Language Explanations

Explaining high-dimensional text classifiers

A Novel Perspective to Look at Attention: Bi-level Attention-based Explainable Topic Modeling for News Classification

Seeing in Words: Learning to Classify through Language Bottlenecks

Interpretable Text Classification Using CNN and Max-pooling

Multidimensional Perceptron for Efficient and Explainable Long Text Classification

Multi-resolution Interpretation and Diagnostics Tool for Natural Language Classifiers

Representing visual classification as a linear combination of words

A Unified Understanding of Deep NLP Models for Text Classification

Explicit Interaction Model towards Text Classification

On Exploring Attention-based Explanation for Transformer Models in Text Classification