Abstract:Natural language understanding (NLU) using neural network pipelines often requires additional context that is not solely present in the input data. Through Prior research, it has been evident that NLU benchmarks are susceptible to manipulation by neural models, wherein these models exploit statistical artifacts within the encoded external knowledge to artificially inflate performance metrics for downstream tasks. Our proposed approach, known as the Recap, Deliberate, and Respond (RDR) paradigm, addresses this issue by incorporating three distinct objectives within the neural network pipeline. Firstly, the Recap objective involves paraphrasing the input text using a paraphrasing model in order to summarize and encapsulate its essence. Secondly, the Deliberation objective entails encoding external graph information related to entities mentioned in the input text, utilizing a graph embedding model. Finally, the Respond objective employs a classification head model that utilizes representations from the Recap and Deliberation modules to generate the final prediction. By cascading these three models and minimizing a combined loss, we mitigate the potential for gaming the benchmark and establish a robust method for capturing the underlying semantic patterns, thus enabling accurate predictions. To evaluate the effectiveness of the RDR method, we conduct tests on multiple GLUE benchmark tasks. Our results demonstrate improved performance compared to competitive baselines, with an enhancement of up to 2\% on standard metrics. Furthermore, we analyze the observed evidence for semantic understanding exhibited by RDR models, emphasizing their ability to avoid gaming the benchmark and instead accurately capture the true underlying semantic patterns.

DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction

Towards Interpreting Recurrent Neural Networks Through Probabilistic Abstraction

Residual Recurrent Neural Networks for Learning Sequential Representations.

DeepCover: Advancing RNN test coverage and online error prediction using state machine extraction

Understanding Hidden Memories of Recurrent Neural Networks

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models

On Attribution of Recurrent Neural Network Predictions via Additive Decomposition

Interpretable extractive text summarization with meta-learning and BI-LSTM: A study of meta learning and explainability techniques

State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictions

Explainable AI: A Brief Survey on History, Research Areas, Approaches and Challenges

Automated Natural Language Explanation of Deep Visual Neurons with Large Models

Interpreting Deep Learning Model Using Rule-based Method

Explaining the Deep Natural Language Processing by Mining Textual Interpretable Features

Explainable machine learning by SEE-Net: closing the gap between interpretable models and DNNs

Interpreting Deep Learning Models for Knowledge Tracing

RDR: the Recap, Deliberate, and Respond Method for Enhanced Language Understanding

Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data

Understanding Spending Behavior: Recurrent Neural Network Explanation and Interpretation

MEME: Generating RNN Model Explanations via Model Extraction