Abstract:Dual process theory posits that human cognition arises via two systems. System 1, which is a quick, emotional, and intuitive process, which is subject to cognitive biases, and System 2, is a slow, onerous, and deliberate process. NLP researchers often compare zero-shot prompting in LLMs to System 1 reasoning and chain-of-thought (CoT) prompting to System 2. In line with this interpretation, prior research has found that using CoT prompting in LLMs leads to reduced gender bias. We investigate the relationship between bias, CoT prompting, a debiasing prompt, and dual process theory in LLMs directly. We compare zero-shot CoT, debiasing, and a variety of dual process theory-based prompting strategies on two bias datasets spanning nine different social bias categories. We incorporate human and machine personas to determine whether the effects of dual process theory in LLMs exist independent of explicit persona models or are based on modeling human cognition. We find that a human persona, debiasing, System 2, and CoT prompting all tend to reduce social biases in LLMs, though the best combination of features depends on the exact model and bias category -- resulting in up to a 19 percent drop in stereotypical judgments by an LLM.

What problem does this paper attempt to address?

The paper attempts to address the issue of how to reduce social bias in large language models (LLMs) through System 1 and System 2 cognitive processes. Specifically, the researchers explore the effects of different prompting techniques (including chain-of-thought prompts, debiasing prompts, etc.) combined with dual-process theory to reduce the performance of LLMs in handling social bias. ### Background and Motivation 1. **Development of Large Language Models**: In recent years, large language models such as GPT-4, ChatGPT, Llama 2, etc., have made significant advancements both technically and socially. These models can not only mimic human language but also demonstrate deep understanding capabilities. 2. **Social Bias Issue**: Despite their excellent performance in many tasks, these models still exhibit embedded social biases, raising ethical concerns about their use in real-world applications. 3. **Dual-Process Theory**: The dual-process theory in psychology divides human cognition into the fast, emotional, intuitive System 1 and the slow, effortful, deliberative System 2. Researchers attempt to apply this theory to LLMs to explore methods for reducing social bias. ### Research Methods 1. **Datasets**: The researchers used two bias detection datasets, StereoSet and GenAssocBias, covering multiple categories of social bias, including age, appearance, gender, institution, nationality, occupation, race, and religion. 2. **Experimental Setup**: The researchers evaluated five major language models (GPT-4, GPT-3.5, Llama 2, Mistral 7B, Gemini) using 12 different prompting techniques, including chain-of-thought prompts, System 1 and System 2 prompts, human and machine role prompts, and combined them with 6 explicit debiasing prompt techniques. 3. **Prompting Techniques**: - **Standard Prompt**: Direct questioning without any special instructions. - **Chain-of-Thought Prompt**: Encouraging the model to think step by step. - **System 1 Prompt**: Answering questions quickly and intuitively. - **System 2 Prompt**: Answering questions slowly and deliberatively. - **Human and Machine Role Prompt**: Having the model play human or machine roles to answer questions differently. - **Debiasing Prompt**: Explicitly instructing the model to avoid stereotypes. ### Key Findings 1. **Best Combination of Human Role and System 2 Prompt**: The combination of human role and System 2 prompt significantly reduced social bias, especially in gender, appearance, and occupation biases. 2. **Chain-of-Thought Prompt Did Not Reduce Bias**: Contrary to expectations, the chain-of-thought prompt did not reduce social bias and even increased the response rate of stereotypes in some cases. 3. **Importance of Human Role**: Adding a human role significantly reduced social bias in both System 1 and System 2 prompts. 4. **Model and Bias Category Specific Effects**: Different models and bias categories responded differently to prompting techniques, but overall, the combination of human role and System 2 prompt performed the best. ### Conclusion The study successfully reduced social bias in large language models by combining dual-process theory with various prompting techniques. In particular, the combination of human role and System 2 prompt proved to be the most effective debiasing method. These findings provide important references for future research and practical applications.

Prompting Techniques for Reducing Social Bias in LLMs through System 1 and System 2 Cognitive Processes

Cognitive Bias in Decision-Making with LLMs

Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Models

Breaking Bias, Building Bridges: Evaluation and Mitigation of Social Biases in LLMs via Contact Hypothesis

Can Instruction Fine-Tuned Language Models Identify Social Bias through Prompting?

Steering LLMs Towards Unbiased Responses: A Causality-Guided Debiasing Framework

Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustment

Social Bias Evaluation for Large Language Models Requires Prompt Variations

Reasoning Beyond Bias: A Study on Counterfactual Prompting and Chain of Thought Reasoning

Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting

Argumentative Experience: Reducing Confirmation Bias on Controversial Issues through LLM-Generated Multi-Persona Debates

Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions

Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs

A Multi-LLM Debiasing Framework

Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting

Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models

Measuring Implicit Bias in Explicitly Unbiased Large Language Models

Unboxing Occupational Bias: Grounded Debiasing of LLMs with U.S. Labor Data

Do LLMs exhibit human-like response biases? A case study in survey design

Challenging the appearance of machine intelligence: Cognitive bias in LLMs and Best Practices for Adoption

Social Debiasing for Fair Multi-modal LLMs