Abstract:Large language models (LLMs) closely interact with humans, and thus need an intimate understanding of the cultural values of human society. In this paper, we explore how open-source LLMs make judgments on diverse categories of cultural values across countries, and its relation to training methodology such as model sizes, training corpus, alignment, etc. Our analysis shows that LLMs can judge socio-cultural norms similar to humans but less so on social systems and progress. In addition, LLMs tend to judge cultural values biased toward Western culture, which can be improved with training on the multilingual corpus. We also find that increasing model size helps a better understanding of social values, but smaller models can be enhanced by using synthetic data. Our analysis reveals valuable insights into the design methodology of LLMs in connection with their understanding of cultural values.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to explore the performance of large - language models (LLMs) in cross - cultural value judgment and their relationship with training methods. Specifically, the research aims to answer the following key questions: 1. **How do LLMs understand and judge the cultural values of different countries?** - The research finds that the judgments of LLMs in terms of sociocultural norms are similar to those of humans, but their judgments in terms of social systems and progress are less consistent. 2. **Are there biases in the cultural value judgments of LLMs?** - The results show that the judgments of LLMs tend to be biased towards Western culture, which may be because they are mainly trained on English corpora. 3. **How can the understanding of different cultures by LLMs be improved by improving training methods?** - The research points out that training with multilingual corpora can significantly improve the understanding of non - Western cultures by LLMs. - Larger models show better performance in understanding cultural values. - Synthetic data can help smaller models improve their cultural understanding ability. - Alignment technology can make LLMs closer to human cultural judgments. ### Summary of main research findings: - **Sociocultural norms**: The judgments of LLMs in terms of sociocultural norms are similar to those of humans, but their judgments in terms of social systems and progress are less consistent. - **Cultural bias**: The judgments of LLMs tend to be biased towards Western culture, but this situation can be improved by training with multilingual corpora. - **Model scale**: Larger models perform better in understanding cultural values. - **Synthetic data**: Synthetic data can enhance the cultural understanding ability of smaller models. - **Alignment technology**: Alignment technology enables LLMs to better imitate human cultural judgments. Through these findings, the research provides important insights for the design and improvement of LLMs, especially in cross - cultural communication and understanding.

Exploring Large Language Models on Cross-Cultural Values in Connection with Training Methodology

Investigating Cultural Alignment of Large Language Models

Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions

Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning

CultureLLM: Incorporating Cultural Differences into Large Language Models

How well do llms represent values across cultures? empirical analysis of llm responses based on hofstede cultural dimensions

Methodology of Adapting Large English Language Models for Specific Cultural Contexts

An Investigation into Value Misalignment in LLM-Generated Texts for Cultural Heritage

Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches

LLMs as mirrors of societal moral standards: reflection of cultural divergence and agreement across ethical topics

CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models

Cultural Fidelity in Large-Language Models: An Evaluation of Online Language Resources as a Driver of Model Performance in Value Representation

Socially Responsible Data for Large Multilingual Language Models

Large Language Models as Superpositions of Cultural Perspectives

X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment

The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models

Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense

Measuring Spiritual Values and Bias of Large Language Models

From Values to Opinions: Predicting Human Behaviors and Stances Using Value-Injected Large Language Models

From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models