Abstract:Large language models (LLMs) offer significant potential as tools to support an expanding range of decision-making tasks. Given their training on human (created) data, LLMs have been shown to inherit societal biases against protected groups, as well as be subject to bias functionally resembling cognitive bias. Human-like bias can impede fair and explainable decisions made with LLM assistance. Our work introduces BiasBuster, a framework designed to uncover, evaluate, and mitigate cognitive bias in LLMs, particularly in high-stakes decision-making tasks. Inspired by prior research in psychology and cognitive science, we develop a dataset containing 13,465 prompts to evaluate LLM decisions on different cognitive biases (e.g., prompt-induced, sequential, inherent). We test various bias mitigation strategies, while proposing a novel method utilizing LLMs to debias their own human-like cognitive bias within prompts. Our analysis provides a comprehensive picture of the presence and effects of cognitive bias across commercial and open-source models. We demonstrate that our selfhelp debiasing effectively mitigates model answers that display patterns akin to human cognitive bias without having to manually craft examples for each bias.

What problem does this paper attempt to address?

The paper attempts to address the issue of human cognitive biases exhibited by large language models (LLMs) in decision support tasks. Specifically: 1. **Identification and Evaluation of Cognitive Biases**: The paper evaluates the performance of LLMs on different cognitive biases (such as prompt-induced, sequential, and intrinsic biases) by constructing a dataset containing 13,465 prompts. 2. **Mitigation of Cognitive Biases**: The paper proposes a framework named BIASBUSTER, aimed at discovering, evaluating, and mitigating cognitive biases in LLMs, especially in high-stakes decision-making tasks. This framework includes various bias mitigation strategies, such as zero-shot prompting, few-shot prompting, and self-help methods. ### Main Research Content - **Background and Motivation**: LLMs perform excellently in various tasks, but they inherit social biases from training data and may exhibit human-like cognitive biases, affecting their fairness and interpretability in decision support tasks. - **Research Methods**: - **Dataset Construction**: Developed a dataset containing 13,465 prompts to evaluate the performance of LLMs on different cognitive biases. - **Evaluation Metrics**: Designed various evaluation metrics, such as normalized Euclidean distance, to measure the consistency of decisions made by the model under different sequences. - **Bias Mitigation Strategies**: Tested various bias mitigation strategies, including zero-shot prompting, few-shot prompting, and self-help methods. - **Experimental Setup**: Evaluated the performance of different models on cognitive biases by simulating high-stakes decision scenarios, such as student admissions. ### Main Contributions - **BIASBUSTER Framework**: Proposed a systematic framework for quantifying and mitigating cognitive biases in LLMs. - **Dataset**: Provided a dataset containing 13,465 prompts, which can be used to test the performance of LLMs on cognitive biases. - **Evaluation and Mitigation Strategies**: Demonstrated how to effectively reduce cognitive biases in LLMs through various evaluation metrics and mitigation strategies. ### Experimental Results - **Sequential Bias**: Observed that LLMs exhibit lower decision confidence in the original (random sequence) evaluation setting, possibly due to the influence of prior decisions and unawareness of biases. - **Prompt-Induced Bias**: For framing bias and group attribution bias, all models exhibited different behaviors under different settings, indicating decision inconsistency when faced with different frames and gender attributes. - **Mitigation Effectiveness**: Most mitigation strategies improved model performance to some extent, with self-help methods showing better effectiveness across multiple models. In summary, by introducing the BIASBUSTER framework and the related dataset, the paper systematically evaluates and mitigates cognitive biases in LLMs, providing important tools and methods to enhance the fairness and reliability of LLMs in decision support tasks.

Cognitive Bias in Decision-Making with LLMs

A Comprehensive Evaluation of Cognitive Biases in LLMs

CBEval: A framework for evaluating and interpreting cognitive biases in LLMs

Evaluating and Mitigating Social Bias for Large Language Models in Open-ended Settings

Balancing Rigor and Utility: Mitigating Cognitive Biases in Large Language Models for Multiple-Choice Questions

Breaking Bias, Building Bridges: Evaluation and Mitigation of Social Biases in LLMs via Contact Hypothesis

Steering LLMs Towards Unbiased Responses: A Causality-Guided Debiasing Framework

How Can We Diagnose and Treat Bias in Large Language Models for Clinical Decision-Making?

Measuring Implicit Bias in Explicitly Unbiased Large Language Models

Evaluation and mitigation of cognitive biases in medical language models

Evaluating Nuanced Bias in Large Language Model Free Response Answers

Bias patterns in the application of LLMs for clinical decision support: A comprehensive study

Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models

A Multi-LLM Debiasing Framework

Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective

Benchmarking Cognitive Biases in Large Language Models as Evaluators

BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMs

Addressing cognitive bias in medical language models

Do LLMs exhibit human-like response biases? A case study in survey design

An Actionable Framework for Assessing Bias and Fairness in Large Language Model Use Cases

Unboxing Occupational Bias: Grounded Debiasing of LLMs with U.S. Labor Data