What Is Being Argued (WIBA)? An Application to Legislative Deliberation in the U.S. Congress

Arman Irani,Ju Yeon Park,Kevin Esterling,Michalis Faloutsos

2024-07-16

Abstract:How can we utilize state-of-the-art NLP tools to better understand legislative deliberation? Committee hearings are a core feature of any legislature, and they offer an institutional setting which promotes the exchange of arguments and reasoning that directly impact and shape legislation. We apply What Is Being Argued (WIBA), which is an argument extraction and analysis framework that we previously developed, to U.S. Congressional committee hearings from 2005 to 2023 (109th to 117th Congresses). Then, we further expand WIBA by introducing new ways to quantify various dynamics of democratic deliberation. Specifically, these extensions present a variety of summary statistics capturing how deliberative or controversial a discourse was, as well as useful visualizations to the WIBA output that aid analyzing arguments made during the legislative deliberation. Our application reveals potential biases in the committee system, and how political parties control the flow of information in 'hot topic' hearings.

Social and Information Networks

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to use the most advanced natural language processing (NLP) tools to better understand the legislative deliberation process in the U.S. Congressional committee hearings. Specifically, the paper focuses on how to analyze a large amount of text data to reveal the argumentation and reasoning processes in these hearings, so as to gain a deeper understanding of the quality and characteristics of democratic deliberation.

### Problem Background

Political meetings usually generate a large amount of dialog text, which records the process of legislative deliberation. The U.S. Congressional committee hearings are the core part of policy - making, with thousands of hearings held every year, producing a vast amount of text data. However, currently, the application of computational text analysis methods to these texts is relatively scarce, making it difficult to effectively understand and evaluate their deliberation quality.

### Solution

To meet this challenge, the authors propose to use the WIBA (What Is Being Argued) framework they previously developed to analyze the argumentation and reasoning in the U.S. Congressional committee hearings. WIBA is an argument - centered NLP framework that can perform the following three tasks:

1. **Identify Arguments**: Determine whether there are arguments in a piece of text.
2. **Extract Topics**: Identify the topics being discussed.
3. **Detect Stances**: Recognize the supporting or opposing stances of the arguments towards the topics.

On this basis, the paper further extends the functions of WIBA and introduces new quantification and visualization methods to better understand the deliberation dynamics in the hearings. Specifically, these include:

- **Argumentativeness Measure**: Calculate the proportion of arguments at the statement, speaker, and hearing levels.
- **Topic Similarity**: Use the Sentence Transformer model to measure the similarity between arguments and cluster similar arguments.
- **Deliberation Intensity Score (DIS)**: Combine argument diversity (the ratio of the number of clusters to the number of arguments) and argumentativeness measure to quantify the intensity of deliberation.
- **Controversy Measure**: Calculate the difference in the number of arguments for and against a certain topic to quantify the degree of controversy in the discussion.

### Main Contributions

1. **Quantify Argumentativeness**: Propose a method for calculating argumentativeness measure to evaluate the argument density in speakers and hearings.
2. **Visualization Tools**: Develop an interactive dashboard that can intuitively display the argument dynamics in the hearings and help researchers understand the flow and change of arguments.
3. **Deliberation Intensity Score**: Introduce a new indicator - the Deliberation Intensity Score (DIS) - to quantify the diversity and intensity of deliberation.
4. **Controversy Analysis**: Quantify the degree of controversy in the discussion by calculating the difference in the number of arguments for and against a certain topic.

### Case Studies

The paper demonstrates the effectiveness of the WIBA framework and its extended methods through empirical analysis of hearings involving abortion and genetically modified organisms (GMOs). The results show that the deliberation intensity in the abortion hearings is significantly higher than that in the GMO hearings, and the Republicans tend to invite more witnesses who share their views when they control the majority. In addition, the question - and - answer sessions in the abortion hearings reveal more new information, while there is more information repetition in the GMO hearings.

### Conclusion

This paper demonstrates the potential applications of the WIBA framework in analyzing U.S. Congressional legislative deliberations and proposes a variety of useful quantification and visualization methods, which are helpful for future research on the effectiveness of legislative deliberation systems, the behavior of legislators, the influence of external groups on the legislative process, and linguistic inquiries. These methods are not only applicable to the U.S. Congressional hearings but can also be extended to other institutional settings where dialog text needs to be analyzed.