Abstract:Proper moral beliefs are fundamental for language models, yet assessing these beliefs poses a significant challenge. This study introduces a novel three-module framework to evaluate the moral beliefs of four prominent large language models. Initially, we constructed a dataset containing 472 moral choice scenarios in Chinese, derived from moral words. The decision-making process of the models in these scenarios reveals their moral principle preferences. By ranking these moral choices, we discern the varying moral beliefs held by different language models. Additionally, through moral debates, we investigate the firmness of these models to their moral choices. Our findings indicate that English language models, namely ChatGPT and Gemini, closely mirror moral decisions of the sample of Chinese university students, demonstrating strong adherence to their choices and a preference for individualistic moral beliefs. In contrast, Chinese models such as Ernie and ChatGLM lean towards collectivist moral beliefs, exhibiting ambiguity in their moral choices and debates. This study also uncovers gender bias embedded within the moral beliefs of all examined language models. Our methodology offers an innovative means to assess moral beliefs in both artificial and human intelligence, facilitating a comparison of moral values across different cultures.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to evaluate the moral beliefs in large language models (LLMs) and explore the moral decision - making capabilities of these models in different cultural contexts. Specifically, the researchers constructed a data set containing 472 Chinese moral choice scenarios, through which they evaluated the moral beliefs of four well - known large language models. The research methods include three modules: moral choice, moral ranking, and moral debate. Through these modules, the researchers were able to reveal the preferences of different language models in moral decision - making and their firmness when facing moral challenges. In addition, the study also found the phenomenon of gender bias embedded in the moral beliefs of these models. ### Main research questions: 1. **Evaluating the moral beliefs of LLMs**: Researchers hope to evaluate the performance of LLMs in moral decision - making through the constructed data set of moral choice scenarios and understand their preferences for moral principles. 2. **Comparing the differences in moral beliefs of LLMs in different cultural contexts**: The study specifically focuses on the differences in moral decision - making between English LLMs (such as ChatGPT and Gemini) and Chinese LLMs (such as Ernie and ChatGLM), and explores the impact of cultural context on the moral beliefs of models. 3. **Evaluating the firmness of LLMs in moral debates**: By setting up a moral debate session, the researchers examined whether the moral choices of LLMs would change when facing different viewpoints and their firmness in moral decision - making. 4. **Discovering and analyzing gender bias**: The study also explored how gender factors affect the moral decision - making of LLMs and revealed the phenomenon of gender bias existing in the models. ### Research methods: 1. **Moral choice**: Through the constructed 472 moral choice scenarios, let LLMs make moral decisions and record their choice results. 2. **Moral ranking**: Use Best - Worst Scaling (BWS) and Iterative Luce Spectral Ranking (ILSR) methods to rank the moral choices of LLMs and reveal the priority of their moral principles. 3. **Moral debate**: By simulating moral debates, examine the reactions of LLMs when facing different viewpoints and evaluate the stability of their moral choices. ### Main findings: - **The impact of cultural context on moral beliefs**: English LLMs (such as ChatGPT and Gemini) are more inclined to individualistic moral beliefs, while Chinese LLMs (such as Ernie and ChatGLM) are more inclined to collectivist moral beliefs. - **Gender bias**: The study found that all evaluated LLMs have gender bias, indicating that the models may inherit and reinforce the stereotypes in the real world. - **The firmness of moral choices**: Through multi - round debates, the study evaluated the firmness of LLMs when facing moral challenges and found that different models showed significant differences in this aspect. ### Significance: - **Expanding the scope of moral research**: By using Chinese as the research language, it expands the previous moral research mainly based on English and reveals the differences in moral beliefs of LLMs in different cultural contexts. - **Improving moral alignment**: By identifying and understanding the moral beliefs and biases of LLMs, the moral decision - making of models can be better aligned, reducing potential moral risks. - **Promoting the stability and reliability of models**: Through multi - round debates, the stability of LLMs in moral decision - making is evaluated, which helps to improve the reliability and trustworthiness of models in practical applications.

Evaluating Moral Beliefs across LLMs through a Pluralistic Framework

Evaluating the Moral Beliefs Encoded in LLMs

Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experiment

CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models

Exploring and steering the moral compass of Large Language Models

Moral Foundations of Large Language Models

The Moral Mind(s) of Large Language Models

MoralBench: Moral Evaluation of LLMs

LLMs as mirrors of societal moral standards: reflection of cultural divergence and agreement across ethical topics

Speaking Multiple Languages Affects the Moral Bias of Language Models

Evaluating Gender Bias of LLMs in Making Morality Judgements

Language Model Alignment in Multilingual Trolley Problems

Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in

Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations

Large Language Models as Mirrors of Societal Moral Standards

Exploring the psychology of LLMs' Moral and Legal Reasoning

Large-scale moral machine experiment on large language models

Western, Religious or Spiritual: An Evaluation of Moral Justification in Large Language Models

The moral machine experiment on large language models

The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making