Abstract:Large Language Models (LLMs) are often augmented with external information as contexts, but this external information can sometimes be inaccurate or even intentionally misleading. We argue that robust LLMs should demonstrate situated faithfulness, dynamically calibrating their trust in external information based on their confidence in the internal knowledge and the external context. To benchmark this capability, we evaluate LLMs across several QA datasets, including a newly created dataset called RedditQA featuring in-the-wild incorrect contexts sourced from Reddit posts. We show that when provided with both correct and incorrect contexts, both open-source and proprietary models tend to overly rely on external information, regardless of its factual accuracy. To enhance situated faithfulness, we propose two approaches: Self-Guided Confidence Reasoning (SCR) and Rule-Based Confidence Reasoning (RCR). SCR enables models to self-access the confidence of external information relative to their own internal knowledge to produce the most accurate answer. RCR, in contrast, extracts explicit confidence signals from the LLM and determines the final answer using predefined rules. Our results show that for LLMs with strong reasoning capabilities, such as GPT-4o and GPT-4o mini, SCR outperforms RCR, achieving improvements of up to 24.2% over a direct input augmentation baseline. Conversely, for a smaller model like Llama-3-8B, RCR outperforms SCR. Fine-tuning SCR with our proposed Confidence Reasoning Direct Preference Optimization (CR-DPO) method improves performance on both seen and unseen datasets, yielding an average improvement of 8.9% on Llama-3-8B. In addition to quantitative results, we offer insights into the relative strengths of SCR and RCR. Our findings highlight promising avenues for improving situated faithfulness in LLMs. The data and code are released.

Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding

Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts

Adversarial Contrastive Decoding: Boosting Safety Alignment of Large Language Models via Opposite Prompt Optimization

Supervised Knowledge Makes Large Language Models Better In-context Learners

When to Speak, When to Abstain: Contrastive Decoding with Abstention

Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?

Multimodal Contrastive In-Context Learning

Customizing Language Model Responses with Contrastive In-Context Learning

AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge

Can Large Language Models Understand Context?

Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping

Enhancing and Accelerating Large Language Models via Instruction-Aware Contextual Compression

Context-faithful Prompting for Large Language Models

MAGNIFICo: Evaluating the In-Context Learning Ability of Large Language Models to Generalize to Novel Interpretations

End-to-End Speech Recognition Contextualization with Large Language Models

Large Language Models Know What Makes Exemplary Contexts

Entropy-Based Decoding for Retrieval-Augmented Large Language Models

Enhancing Large Language Models' Situated Faithfulness to External Contexts

Context Matter: Data-Efficient Augmentation of Large Language Models for Scientific Applications

Decoupled Context Processing for Context Augmented Language Modeling

Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding