Representative Social Choice: From Learning Theory to AI Alignment

Tianyi Qiu

2024-10-31

Abstract:Social choice theory is the study of preference aggregation across a population, used both in mechanism design for human agents and in the democratic alignment of language models. In this study, we propose the representative social choice framework for the modeling of democratic representation in collective decisions, where the number of issues and individuals are too large for mechanisms to consider all preferences directly. These scenarios are widespread in real-world decision-making processes, such as jury trials, indirect elections, legislation processes, corporate governance, and, more recently, language model alignment. In representative social choice, the population is represented by a finite sample of individual-issue pairs based on which social choice decisions are made. We show that many of the deepest questions in representative social choice can be naturally formulated as statistical learning problems, and prove the generalization properties of social choice mechanisms using the theory of machine learning. We further formulate axioms for representative social choice, and prove Arrow-like impossibility theorems with new combinatorial tools of analysis. Our framework introduces the representative approach to social choice, opening up research directions at the intersection of social choice, learning theory, and AI alignment.

Machine Learning,Artificial Intelligence,Computation and Language,Computers and Society,Computer Science and Game Theory

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper, "Representative Social Choice: From Learning Theory to AI Alignment", attempts to solve several core problems in social choice theory, especially the problem of how to effectively make democratic representative - based decisions when the number of individuals and the number of issues are extremely large. Specifically, the paper mainly focuses on the following aspects:

1. **The representativeness problem in large - scale decision - making**:
- Existing social choice models usually assume that individuals have clear preferences for each issue and that these preferences can be fully collected and processed. However, in actual large - scale elections or decision - making processes, this assumption often does not hold. Candidates may have multiple inter - related policy positions, and it is almost impossible to collect all voters' preferences on all issues.
- The paper proposes a new framework - **Representative Social Choice**. Under this framework, through a limited sample of individual - issue pairs, the preferences of the entire group are represented to make collective decisions.

2. **The application of statistical learning theory**:
- The author naturally transforms the representative social choice problem into a statistical learning problem and uses tools in machine learning theory (such as VC dimension, generalization bounds, etc.) to analyze the properties of social choice mechanisms. This not only provides theoretical support but also provides guidance for practical applications.

3. **Axioms and impossibility theorems**:
- The paper proposes an axiomatic system for representative social choice and proves new results similar to Arrow's impossibility theorem. These theorems reveal that in representative social choice, certain desirable properties (such as probabilistic Pareto efficiency, weak probabilistic independence, etc.) cannot be satisfied simultaneously, thus providing a theoretical basis for designing effective social choice mechanisms.

4. **Practical applications**:
- The representative social choice framework can be applied to a variety of practical scenarios, including jury trials, the legislative process, corporate governance, and AI alignment. Especially in the field of AI alignment, through representative social choice, AI systems such as language models can be trained to make their decisions consistent with human values.

### Summary

In general, this paper aims to solve the representativeness problem in large - scale decision - making. By introducing methods from statistical learning theory, it proposes a new social choice framework and proves its effectiveness and limitations in theoretical and practical applications. This research not only enriches social choice theory but also provides new ideas for practical applications in fields such as AI alignment.