Abstract:Pre-trained Language models (PLMs) have been acknowledged to contain harmful information, such as social biases, which may cause negative social impacts or even bring catastrophic results in application. Previous works on this problem mainly focused on using black-box methods such as probing to detect and quantify social biases in PLMs by observing model outputs. As a result, previous debiasing methods mainly finetune or even pre-train PLMs on newly constructed anti-stereotypical datasets, which are high-cost. In this work, we try to unveil the mystery of social bias inside language models by introducing the concept of {\sc Social Bias Neurons}. Specifically, we propose {\sc Integrated Gap Gradients (IG$^2$)} to accurately pinpoint units (i.e., neurons) in a language model that can be attributed to undesirable behavior, such as social bias. By formalizing undesirable behavior as a distributional property of language, we employ sentiment-bearing prompts to elicit classes of sensitive words (demographics) correlated with such sentiments. Our IG$^2$ thus attributes the uneven distribution for different demographics to specific Social Bias Neurons, which track the trail of unwanted behavior inside PLM units to achieve interoperability. Moreover, derived from our interpretable technique, {\sc Bias Neuron Suppression (BNS)} is further proposed to mitigate social biases. By studying BERT, RoBERTa, and their attributable differences from debiased FairBERTa, IG$^2$ allows us to locate and suppress identified neurons, and further mitigate undesired behaviors. As measured by prior metrics from StereoSet, our model achieves a higher degree of fairness while maintaining language modeling ability with low cost\footnote{This work contains examples that potentially implicate stereotypes, associations, and other harms that could be offensive to individuals in certain social groups.}.

Deceptively simple: An outsider's perspective on natural language processing

Making the Implicit Explicit: Implicit Content as a First Class Citizen in NLP

Political DEBATE: Efficient Zero-shot and Few-shot Classifiers for Political Text

When in Doubt, Cascade: Towards Building Efficient and Capable Guardrails

Inducing Political Bias Allows Language Models Anticipate Partisan Reactions to Controversies

Banal Deception Human-AI Ecosystems: A Study of People's Perceptions of LLM-generated Deceptive Behaviour

Towards Understanding and Mitigating Social Biases in Language Models

Beyond Hate Speech: NLP's Challenges and Opportunities in Uncovering Dehumanizing Language

A Framework for the Computational Linguistic Analysis of Dehumanization

Balancing Transparency and Accuracy: A Comparative Analysis of Rule-Based and Deep Learning Models in Political Bias Classification

The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Language Models

We Don't Speak the Same Language: Interpreting Polarization through Machine Translation

Unmasking the Shadows of AI: Investigating Deceptive Capabilities in Large Language Models

A Machine Learning Pipeline to Examine Political Bias with Congressional Speeches

Deceptive XAI: Typology, Creation and Detection

Undesirable Biases in NLP: Addressing Challenges of Measurement

Exposing Bias in Online Communities through Large-Scale Language Models

From Complexity to Clarity: How AI Enhances Perceptions of Scientists and the Public's Understanding of Science

KnowBias: A Novel AI Method to Detect Polarity in Online Content

Testing the effectiveness of saliency-based explainability in NLP using randomized survey-based experiments

The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models