Abstract:The rise of digital misinformation has heightened interest in using multilingual Large Language Models (LLMs) for fact-checking. This study systematically evaluates translation bias and the effectiveness of LLMs for cross-lingual claim verification across 15 languages from five language families: Romance, Slavic, Turkic, Indo-Aryan, and Kartvelian. Using the XFACT dataset to assess their impact on accuracy and bias, we investigate two distinct translation methods: pre-translation and self-translation. We use mBERT's performance on the English dataset as a baseline to compare language-specific accuracies. Our findings reveal that low-resource languages exhibit significantly lower accuracy in direct inference due to underrepresentation in the training data. Furthermore, larger models demonstrate superior performance in self-translation, improving translation accuracy and reducing bias. These results highlight the need for balanced multilingual training, especially in low-resource languages, to promote equitable access to reliable fact-checking tools and minimize the risk of spreading misinformation in different linguistic contexts.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problems of translation bias and accuracy in multilingual large language models (LLMs) in cross - language claim verification tasks. Specifically, the research focuses on the following aspects: 1. **Translation Bias and Accuracy Evaluation**: - Researchers systematically evaluated the translation bias and effectiveness of LLMs in 15 different languages (covering five language families: Romance, Slavic, Turkic, Indo - Aryan, and Kartvelian). - The XFACT dataset was used to evaluate the impact of these biases on accuracy and fairness. 2. **Performance in Low - Resource Languages**: - The study found that low - resource languages (such as Kannada and Occitan) showed significantly lower accuracy in direct reasoning, mainly due to under - representation in the training data. - It emphasized the importance of balanced multilingual training, especially in low - resource languages, to promote fair access to reliable fact - checking tools and reduce the risk of spreading misinformation in different language environments. 3. **Comparison of Translation Methods**: - The study compared two different translation methods: pre - translation and self - translation. The results showed that larger models performed better in self - translation, improving translation accuracy and reducing bias. 4. **Impact of Model Scale**: - As the model scale increased, the accuracy of self - translation was significantly improved, but among all models, the translation bias remained consistent, indicating that larger models do not necessarily ensure fairness among languages. ### Summary This research, by evaluating the performance of multilingual large language models in cross - language claim verification, revealed the challenges in low - resource languages and emphasized the need to develop more balanced and fair language technologies. By identifying translation biases in specific areas, researchers laid the foundation for improving the balance of multilingual capabilities.

A Comparative Study of Translation Bias and Accuracy in Multilingual Large Language Models for Cross-Language Claim Verification

Do Multilingual Large Language Models Mitigate Stereotype Bias?

Are Large Language Models Good Fact Checkers: A Preliminary Study

Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong

Bias and Fairness in Large Language Models: A Survey

Could We Have Had Better Multilingual LLMs If English Was Not the Central Language?

Bias Similarity Across Large Language Models

Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Research

Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models

Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals

Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformation

Multimodal Large Language Models to Support Real-World Fact-Checking

A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias

Towards Resource Efficient and Interpretable Bias Mitigation in Large Language Models

Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models

The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models

People's Perceptions Toward Bias and Related Concepts in Large Language Models: A Systematic Review

Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Media

Lost in Translation -- Multilingual Misinformation and its Evolution

Evaluating and Mitigating Linguistic Discrimination in Large Language Models