Abstract:This paper examines how large language models (LLMs) can help people write constructive comments in online debates on divisive social issues and whether the notions of constructiveness vary across cultures. Through controlled experiments with 600 participants from India and the US, who reviewed and wrote constructive comments on online threads on Islamophobia and homophobia, we found potential misalignment in how LLMs and humans perceive constructiveness in online comments. While the LLM was more likely to view dialectical comments as more constructive, participants favored comments that emphasized logic and facts more than the LLM did. Despite these differences, participants rated LLM-generated and human-AI co-written comments as significantly more constructive than those written independently by humans. Our analysis also revealed that LLM-generated and human-AI co-written comments exhibited more linguistic features associated with constructiveness compared to human-written comments on divisive topics. When participants used LLMs to refine their comments, the resulting comments were longer, more polite, positive, less toxic, and more readable, with added argumentative features that retained the original intent but occasionally lost nuances. Based on these findings, we discuss ethical and design considerations in using LLMs to facilitate constructive discourse online.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to explore how large - language models (LLMs) can help people write constructive comments in online debates on socially divisive topics, and to study whether there are differences in people's perception of constructive comments across different cultural backgrounds. Specifically, the paper attempts to answer the following three research questions: 1. **Are there differences in the perception of constructive comments between humans and LLMs?** - The research found that, compared with humans, LLMs are more likely to consider dialectical comments more constructive than logical comments. LLMs attach more importance to politeness and balancing different viewpoints, while participants prefer comments that emphasize logic and facts. 2. **Can LLMs help people write more constructive comments on socially divisive topics?** - The experimental results show that both the comments generated by LLMs and the comments written by human - machine collaboration are significantly more constructive than those written entirely independently by humans. These comments are longer, more polite, more positive, less toxic and more readable, while increasing argumentative features, retaining the original intention but sometimes losing some nuances. 3. **Are there differences in people's perception of the constructiveness of online comments across different cultural backgrounds?** - The research found that participants from India and the United States both consider dialectical comments more constructive than logical comments and provide similar reasons. They show comparable skills in writing constructive comments independently or with the assistance of LLMs. In the test group, participants from both countries use similar prompts to make their comments more constructive, and they generally believe that the comments generated by LLMs and written by human - machine collaboration are more constructive than those written independently by humans. Through the research of these questions, the paper not only provides quantitative and qualitative evidence indicating that LLMs can help people from different cultural backgrounds write constructive comments on socially divisive topics, but also reveals the potential inconsistency between humans and LLMs in different argumentative styles when evaluating constructiveness, and discusses the ethical and design issues that need to be considered when designing socio - technical systems to promote constructive discussions on cross - cultural socially divisive topics.

Examining Human-AI Collaboration for Co-Writing Constructive Comments Online

Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits

Can AI Moderate Online Communities?

Collective Constitutional AI: Aligning a Language Model with Public Input

Limits of Large Language Models in Debating Humans

LLMs as Debate Partners: Utilizing Genetic Algorithms and Adversarial Search for Adaptive Arguments

Evaluating LLMs Capabilities Towards Understanding Social Dynamics

AI Chat Assistants can Improve Conversations about Divisive Topics

Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers

Argumentative Experience: Reducing Confirmation Bias on Controversial Issues through LLM-Generated Multi-Persona Debates

Exploring the Impact of Artificial Intelligence-Mediated Communication on Bias and Information Loss in Non-academic and Academic Writing Contexts

Examining the Role of Relationship Alignment in Large Language Models

Potential Benefits of Employing Large Language Models in Research in Moral Education and Development

LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback

Large Language Models as Instruments of Power: New Regimes of Autonomous Manipulation and Control

Leveraging AI for democratic discourse: Chat interventions can improve online political conversations at scale

Vox Populi, Vox ChatGPT: Large Language Models, Education and Democracy

LLMs Among Us: Generative AI Participating in Digital Discourse

LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play

Toward Cultural Interpretability: A Linguistic Anthropological Framework for Describing and Evaluating Large Language Models (LLMs)

Human-AI Collaborative Essay Scoring: A Dual-Process Framework with LLMs