Abstract:Models trained with counterfactually augmented data learn representations of the causal structure of tasks, enabling robust generalization. However, high-quality counterfactual data is scarce for most tasks and not easily generated at scale. When crowdsourced, such data is typically limited in scale and diversity; when generated using supervised methods, it is computationally expensive to extend to new counterfactual dimensions. In this work, we introduce DISCO (DIStilled COunterfactual Data), a new method for automatically generating high quality counterfactual data at scale. DISCO engineers prompts to generate phrasal perturbations with a large general language model. Then, a task-specific teacher model filters these generations to distill high-quality counterfactual data. While task-agnostic, we apply our pipeline to the task of natural language inference (NLI) and find that on challenging evaluations such as the NLI stress test, comparatively smaller student models trained with DISCO generated counterfactuals are more robust (6% absolute) and generalize better across distributions (2%) compared to models trained without data augmentation. Furthermore, DISCO augmented models are 10% more consistent between counterfactual pairs on three evaluation sets, demonstrating that DISCO augmentation enables models to more reliably learn causal representations. Our repository is available at: <a class="link-external link-https" href="https://github.com/eric11eca/disco" rel="external noopener nofollow">this https URL</a>

Relation-based Counterfactual Data Augmentation and Contrastive Learning for Robustifying Natural Language Inference Models

Counterfactual Adversarial Learning with Representation Interpolation

Counterfactual Contrastive Learning for Robust Text Classification

Mitigating Spurious Correlations in Named Entity Recognition Models Through Counterfactual Data Augmentation

Implicit Counterfactual Data Augmentation for Robust Learning

Improving the Natural Language Inference robustness to hard dataset by data augmentation and preprocessing

Inversive-Reasoning Augmentation for Natural Language Inference.

Data Augmentations for Improved (Large) Language Model Generalization

Counterfactual Data Augmentation with Contrastive Learning

Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals

Learning to Contrast the Counterfactual Samples for Robust Visual Question Answering

DISCO: Distilling Counterfactuals with Large Language Models

Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis.

KACE: Generating Knowledge-Aware Contrastive Explanations for Natural Language Inference

EXPLAIN, EDIT, GENERATE: Rationale-Sensitive Counterfactual Data Augmentation for Multi-hop Fact Verification

Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition

Improving Commonsense Causal Reasoning by Adversarial Training and Data Augmentation

PairCFR: Enhancing Model Training on Paired Counterfactually Augmented Data through Contrastive Learning

Reinforced Counterfactual Data Augmentation for Dual Sentiment Classification.

Pseudo dense counterfactual augmentation for aspect-based sentiment analysis

Implicit Counterfactual Data Augmentation for Deep Neural Networks.