Abstract:Explaining the predictions of AI models is paramount in safety-critical applications, such as in legal or medical domains. One form of explanation for a prediction is an extractive rationale, i.e., a subset of features of an instance that lead the model to give its prediction on that instance. For example, the subphrase “he stole the mobile phone” can be an extractive rationale for the prediction of “Theft”. Previous works on generating extractive rationales usually employ a two-phase model: a selector that selects the most important features (i.e., the rationale) followed by a predictor that makes the prediction based exclusively on the selected features. One disadvantage of these works is that the main signal for learning to select features comes from the comparison of the answers given by the predictor to the ground-truth answers. In this work, we propose to squeeze more information from the predictor via an information calibration method. More precisely, we train two models jointly: one is a typical neural model that solves the task at hand in an accurate but black-box manner, and the other is a selector-predictor model that additionally produces a rationale for its prediction. The first model is used as a guide for the second model. We use an adversarial technique to calibrate the information extracted by the two models such that the difference between them is an indicator of the missed or over-selected features. In addition, for natural language tasks, we propose a language-model-based regularizer to encourage the extraction of fluent rationales. Experimental results on a sentiment analysis task, a hate speech recognition task, as well as on three tasks from the legal domain show the effectiveness of our approach to rationale extraction.

Beyond Accuracy: Ensuring Correct Predictions With Correct Rationales

What to Learn, and How: Toward Effective Learning from Rationales

Exploring Distantly-Labeled Rationales in Neural Network Models

Efficient End-to-End Visual Document Understanding with Rationale Distillation

Convincing Rationales for Visual Question Answering Reasoning

Unsupervised Selective Rationalization with Noise Injection

Boosting the Power of Small Multimodal Reasoning Models to Match Larger Models with Self-Consistency Training

Enhancing the Rationale-Input Alignment for Self-explaining Rationalization

Improving Language Model Reasoning with Self-motivated Learning

Learning Credible Deep Neural Networks with Rationale Regularization

Being Right for Whose Right Reasons?

Making a (Counterfactual) Difference One Rationale at a Time

Improving Prototypical Visual Explanations with Reward Reweighing, Reselection, and Retraining

A Comprehensive and Reliable Feature Attribution Method: Double-sided Remove and Reconstruct (DoRaR)

Rationalizing Predictions by Adversarial Information Calibration

DARE: Disentanglement-Augmented Rationale Extraction

Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement

RATIONALYST: Pre-training Process-Supervision for Improving Reasoning

Interpreting Object-level Foundation Models via Visual Precision Search

Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales

Answering Unseen Questions With Smaller Language Models Using Rationale Generation and Dense Retrieval