Abstract:Large Language Models (LLMs) are often augmented with external information as contexts, but this external information can sometimes be inaccurate or even intentionally misleading. We argue that robust LLMs should demonstrate situated faithfulness, dynamically calibrating their trust in external information based on their confidence in the internal knowledge and the external context. To benchmark this capability, we evaluate LLMs across several QA datasets, including a newly created dataset called RedditQA featuring in-the-wild incorrect contexts sourced from Reddit posts. We show that when provided with both correct and incorrect contexts, both open-source and proprietary models tend to overly rely on external information, regardless of its factual accuracy. To enhance situated faithfulness, we propose two approaches: Self-Guided Confidence Reasoning (SCR) and Rule-Based Confidence Reasoning (RCR). SCR enables models to self-access the confidence of external information relative to their own internal knowledge to produce the most accurate answer. RCR, in contrast, extracts explicit confidence signals from the LLM and determines the final answer using predefined rules. Our results show that for LLMs with strong reasoning capabilities, such as GPT-4o and GPT-4o mini, SCR outperforms RCR, achieving improvements of up to 24.2% over a direct input augmentation baseline. Conversely, for a smaller model like Llama-3-8B, RCR outperforms SCR. Fine-tuning SCR with our proposed Confidence Reasoning Direct Preference Optimization (CR-DPO) method improves performance on both seen and unseen datasets, yielding an average improvement of 8.9% on Llama-3-8B. In addition to quantitative results, we offer insights into the relative strengths of SCR and RCR. Our findings highlight promising avenues for improving situated faithfulness in LLMs. The data and code are released.

FRACTAL: Fine-Grained Scoring from Aggregate Text Labels

Fractal-Inspired Sentiment Analysis: Evaluation Of Large Language Models And Deep Learning Methods

Fine-grained LLM Agent: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback

Investigating Automatic Scoring and Feedback using Large Language Models

Wisdom of Instruction-Tuned Language Model Crowds. Exploring Model Label Variation

Parameter-Efficient Instruction Tuning of Large Language Models For Extreme Financial Numeral Labelling

LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback

DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregation

Aligning Large Language Models via Fine-grained Supervision

Fine Tuning LLM for Enterprise: Practical Guidelines and Recommendations

Training Language Models to Generate Text with Citations via Fine-grained Rewards

Learning to Refine with Fine-Grained Natural Language Feedback

LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models

Enhancing Large Language Models' Situated Faithfulness to External Contexts

Enhancing Large Language Model Performance To Answer Questions and Extract Information More Accurately

Modulating Language Model Experiences through Frictions

Speech Recognition Rescoring with Large Speech-Text Foundation Models

Under the Surface: Tracking the Artifactuality of LLM-Generated Data

Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost

Exploring Precision and Recall to assess the quality and diversity of LLMs

Learning Fine-Grained Grounded Citations for Attributed Large Language Models