PoliTune: Analyzing the Impact of Data Selection and Fine-Tuning on Economic and Political Biases in Large Language Models

Ahmed Agiza,Mohamed Mostagir,Sherief Reda

2024-07-28

Abstract:In an era where language models are increasingly integrated into decision-making and communication, understanding the biases within Large Language Models (LLMs) becomes imperative, especially when these models are applied in the economic and political domains. This work investigates the impact of fine-tuning and data selection on economic and political biases in LLMs. In this context, we introduce PoliTune, a fine-tuning methodology to explore the systematic aspects of aligning LLMs with specific ideologies, mindful of the biases that arise from their extensive training on diverse datasets. Distinct from earlier efforts that either focus on smaller models or entail resource-intensive pre-training, PoliTune employs Parameter-Efficient Fine-Tuning (PEFT) techniques, which allow for the alignment of LLMs with targeted ideologies by modifying a small subset of parameters. We introduce a systematic method for using the open-source LLM Llama3-70B for dataset selection, annotation, and synthesizing a preferences dataset for Direct Preference Optimization (DPO) to align the model with a given political ideology. We assess the effectiveness of PoliTune through both quantitative and qualitative evaluations of aligning open-source LLMs (Llama3-8B and Mistral-7B) to different ideologies. Our work analyzes the potential of embedding specific biases into LLMs and contributes to the dialogue on the ethical application of AI, highlighting the importance of deploying AI in a manner that aligns with societal values.

Computation and Language,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

This paper aims to address the issue of bias in the application of large language models (LLMs) in the economic and political domains. Specifically, the researchers developed a method called PoliTune, which adjusts pre-trained LLMs through parameter-efficient fine-tuning (PEFT) techniques to align them with specific political ideologies. This approach differs from previous research that focused on small models or resource-intensive pre-training, as it achieves its goal by modifying only a small portion of the model's parameters. The research team used open-source LLMs (such as Llama3-70B) to generate instruction fine-tuning datasets and preference datasets, and employed direct preference optimization (DPO) techniques to align the model with a given political ideology. They validated the effectiveness of PoliTune through quantitative and qualitative evaluations of two open-source LLMs (Llama3-8B and Mistral-7B) and explored the social implications of deploying LLMs with specific biases in the policy-making process. This work emphasizes the importance of AI ethics applications, particularly in ensuring that AI systems are consistent with societal values.