Abstract:Large Language Models (LLMs) are often augmented with external information as contexts, but this external information can sometimes be inaccurate or even intentionally misleading. We argue that robust LLMs should demonstrate situated faithfulness, dynamically calibrating their trust in external information based on their confidence in the internal knowledge and the external context. To benchmark this capability, we evaluate LLMs across several QA datasets, including a newly created dataset called RedditQA featuring in-the-wild incorrect contexts sourced from Reddit posts. We show that when provided with both correct and incorrect contexts, both open-source and proprietary models tend to overly rely on external information, regardless of its factual accuracy. To enhance situated faithfulness, we propose two approaches: Self-Guided Confidence Reasoning (SCR) and Rule-Based Confidence Reasoning (RCR). SCR enables models to self-access the confidence of external information relative to their own internal knowledge to produce the most accurate answer. RCR, in contrast, extracts explicit confidence signals from the LLM and determines the final answer using predefined rules. Our results show that for LLMs with strong reasoning capabilities, such as GPT-4o and GPT-4o mini, SCR outperforms RCR, achieving improvements of up to 24.2% over a direct input augmentation baseline. Conversely, for a smaller model like Llama-3-8B, RCR outperforms SCR. Fine-tuning SCR with our proposed Confidence Reasoning Direct Preference Optimization (CR-DPO) method improves performance on both seen and unseen datasets, yielding an average improvement of 8.9% on Llama-3-8B. In addition to quantitative results, we offer insights into the relative strengths of SCR and RCR. Our findings highlight promising avenues for improving situated faithfulness in LLMs. The data and code are released.

Local Explanations and Self-Explanations for Assessing Faithfulness in black-box LLMs

Are self-explanations from Large Language Models faithful?

FaithLM: Towards Faithful Explanations for Large Language Models

Evaluating Human Alignment and Model Faithfulness of LLM Rationale

Evaluating the Reliability of Self-Explanations in Large Language Models

Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models

Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models

On Measuring Faithfulness or Self-consistency of Natural Language Explanations

Faithfulness Tests for Natural Language Explanations

Properties and Challenges of LLM-Generated Explanations

Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactuals

The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models

Dissociation of Faithful and Unfaithful Reasoning in LLMs

Evaluating Readability and Faithfulness of Concept-based Explanations

LLM-Generated Black-box Explanations Can Be Adversarially Helpful

Cycles of Thought: Measuring LLM Confidence through Stable Explanations

Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate

From large language models to small logic programs: building global explanations from disagreeing local post-hoc explainers

Can LLMs faithfully generate their layperson-understandable 'self'?: A Case Study in High-Stakes Domains

Enhancing Large Language Models' Situated Faithfulness to External Contexts

In-Context Explainers: Harnessing LLMs for Explaining Black Box Models