Generative AI Voting: Fair Collective Choice is Resilient to LLM Biases and Inconsistencies

Srijoni Majumdar,Edith Elkind,Evangelos Pournaras
2024-09-20
Abstract:Scaling up deliberative and voting participation is a longstanding endeavor -- a cornerstone for direct democracy and legitimate collective choice. Recent breakthroughs in generative artificial intelligence (AI) and large language models (LLMs) unravel new capabilities for AI personal assistants to overcome cognitive bandwidth limitations of humans, providing decision support or even direct representation of human voters at large scale. However, the quality of this representation and what underlying biases manifest when delegating collective decision-making to LLMs is an alarming and timely challenge to tackle. By rigorously emulating with high realism more than >50K LLM voting personas in 81 real-world voting elections, we disentangle the nature of different biases in LLMS (GPT 3, GPT 3.5, and Llama2). Complex preferential ballot formats exhibit significant inconsistencies compared to simpler majoritarian elections that show higher consistency. Strikingly though, by demonstrating for the first time in real-world a proportional representation of voters in direct democracy, we are also able to show that fair ballot aggregation methods, such as equal shares, prove to be a win-win: fairer voting outcomes for humans with fairer AI representation. This novel underlying relationship proves paramount for democratic resilience in progressives scenarios with low voters turnout and voter fatigue supported by AI representatives: abstained voters are mitigated by recovering highly representative voting outcomes that are fairer. These interdisciplinary insights provide remarkable foundations for science, policymakers, and citizens to develop safeguards and resilience for AI risks in democratic innovations.
Artificial Intelligence
What problem does this paper attempt to address?
### What problems does this paper attempt to solve? This paper aims to explore and solve the following key problems: 1. **Cognitive bandwidth limitations in large - scale democratic participation**: - With the development of generative artificial intelligence (Generative AI) and large - language models (LLMs), these technologies can assist or represent human voters in large - scale collective decision - making. However, the quality of such representation and potential biases are urgent problems to be solved. The paper studies the performance of different LLMs (such as GPT 3, GPT 3.5 and Llama2) in different voting scenarios by simulating the performance of more than 50,000 LLM voting roles in 81 real - world elections. 2. **Inconsistencies in complex preference expression formats**: - The paper studies the differences between complex preference expression formats (such as scoring systems, cumulative voting, etc.) and simple - majority elections. The results show that complex preference expression formats exhibit significant inconsistencies, while simple - majority elections are more consistent. 3. **Robustness of fair voting aggregation methods**: - The research finds that fair voting aggregation methods (such as "equal shares") can better deal with AI biases and inconsistencies and provide more fair voting results. In particular, when using fair voting methods, even if a large number of voters are absent, AI representatives can effectively restore the consistency of voting results. 4. **AI representation in low - turnout scenarios**: - When the turnout is low, can AI representatives effectively compensate for the influence of absent voters? The paper shows the performance of AI representatives under different voting aggregation methods by simulating scenarios with different turnouts. The results indicate that in the case of low turnout, AI representatives can help restore a certain degree of voting - result consistency, but the effect depends on the voting method used. 5. **The influence of cognitive biases on the consistency of AI choices**: - The paper also explores how different personal characteristics (such as political beliefs, time - discount factors, etc.) affect the consistency of AI choices. The research shows that some cognitive biases (such as the herd effect) help to improve consistency, while other unconscious biases (such as race and political discussions) may lead to inconsistencies. ### Formula presentation To ensure the correctness and readability of formulas, the following are some formula examples involved in the paper: - **Jaccard similarity** is used to measure the consistency between human and AI choices: \[ J(A, B)=\frac{|A\cap B|}{|A\cup B|} \] where \(A\) and \(B\) are the sets of human and AI choices respectively. - **Shapley value** is used to explain the influence of various human characteristics on the consistency of AI choices: \[ \phi_{i}(v)=\sum_{S\subseteq N\setminus\{i\}}\frac{|S|!(n - |S| - 1)!}{n!}(v(S\cup\{i\})-v(S)) \] where \(\phi_{i}(v)\) represents the Shapley value of feature \(i\), \(v\) is the value function of feature combinations, \(S\) is a feature subset, and \(n\) is the total number of features. ### Summary Through systematic research and empirical analysis, the paper reveals the potential and challenges of generative AI in large - scale democratic participation. In particular, fair voting design can effectively alleviate AI biases and inconsistencies, providing important theoretical and technical support for future digital democracy.