Generative AI Degrades Online Communities

Gordon Burtch,Dokyun Lee,Zhichen Chen

DOI: https://doi.org/10.1145/3624732

IF: 22.7

2024-02-22

Communications of the ACM

Abstract:How large language models are influencing online communities.

computer science, theory & methods, software engineering, hardware & architecture

What problem does this paper attempt to address?

The Consequences of Generative AI for UGC and Online Community Engagement

Gordon Burtch,Dokyun Lee,Zhichen Chen

DOI: https://doi.org/10.2139/ssrn.4521754

2023-01-01

SSRN Electronic Journal

Abstract:Generative AI technologies like ChatGPT are transforming how people create, share, and consume content in various domains. Though these technologies have the potential to democratize information access and streamline content production, they may also have unintended consequences for the health and sustainability of online knowledge communities, particularly those that focus on information exchange and lack robust mechanisms of social attachment. We provide initial evidence of this, examining the impact that ChatGPT has had on user activity at StackOverflow, contrasting with user activity at Reddit. We analyze a large sample of data for a period spanning October 2021 and March 2023, capturing content posted to Stack Overflow and Reddit related to the most popular StackOverflow topics. We find that ChatGPT has led to large, significant declines in StackOverflow questions related to these topics, with larger effects manifesting for topics where ChatGPT is more likely to excel, based on the volume of public, online data that would have been available for training. By contrast, considering Reddit sub-communities focused on the same topics, we find no evidence that ChatGPT has had any effect, suggesting the importance of social attachment in online knowledge communities for community survival. Finally, we consider potential shifts in the quality of answers provided at StackOverflow, considering users' potential reliance on ChatGPT when providing answers, and potential declines in collective expertise as users depart the community. We find that average answer quality has declined significantly, primarily due to user exit. We discuss implications for the management of online knowledge communities.
AI models collapse when trained on recursively generated data

Ilia Shumailov,Zakhar Shumaylov,Yiren Zhao,Nicolas Papernot,Ross Anderson,Yarin Gal

DOI: https://doi.org/10.1038/s41586-024-07566-y

IF: 64.8

Nature

Abstract:Stable diffusion revolutionized image creation from descriptive text. GPT-2 (ref. 1), GPT-3(.5) (ref. 2) and GPT-4 (ref. 3) demonstrated high performance across a variety of language tasks. ChatGPT introduced such language models to the public. It is now clear that generative artificial intelligence (AI) such as large language models (LLMs) is here to stay and will substantially change the ecosystem of online text and images. Here we consider what may happen to GPT-{n} once LLMs contribute much of the text found online. We find that indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. We refer to this effect as 'model collapse' and show that it can occur in LLMs as well as in variational autoencoders (VAEs) and Gaussian mixture models (GMMs). We build theoretical intuition behind the phenomenon and portray its ubiquity among all learned generative models. We demonstrate that it must be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet.
Generative Language Models Exhibit Social Identity Biases

Tiancheng Hu,Yara Kyrychenko,Steve Rathje,Nigel Collier,Sander van der Linden,Jon Roozenbeek

2024-06-17

Abstract:The surge in popularity of large language models has given rise to concerns about biases that these models could learn from humans. We investigate whether ingroup solidarity and outgroup hostility, fundamental social identity biases known from social psychology, are present in 56 large language models. We find that almost all foundational language models and some instruction fine-tuned models exhibit clear ingroup-positive and outgroup-negative associations when prompted to complete sentences (e.g., "We are..."). Our findings suggest that modern language models exhibit fundamental social identity biases to a similar degree as humans, both in the lab and in real-world conversations with LLMs, and that curating training data and instruction fine-tuning can mitigate such biases. Our results have practical implications for creating less biased large-language models and further underscore the need for more research into user interactions with LLMs to prevent potential bias reinforcement in humans.

Computation and Language,Computers and Society
Can AI Moderate Online Communities?

Henrik Axelsen,Johannes Rude Jensen,Sebastian Axelsen,Valdemar Licht,Omri Ross

DOI: https://doi.org/10.48550/arXiv.2306.05122

2023-06-08

Abstract:The task of cultivating healthy communication in online communities becomes increasingly urgent, as gaming and social media experiences become progressively more immersive and life-like. We approach the challenge of moderating online communities by training student models using a large language model (LLM). We use zero-shot learning models to distill and expand datasets followed by a few-shot learning and a fine-tuning approach, leveraging open-access generative pre-trained transformer models (GPT) from OpenAI. Our preliminary findings suggest, that when properly trained, LLMs can excel in identifying actor intentions, moderating toxic comments, and rewarding positive contributions. The student models perform above-expectation in non-contextual assignments such as identifying classically toxic behavior and perform sufficiently on contextual assignments such as identifying positive contributions to online discourse. Further, using open-access models like OpenAI's GPT we experience a step-change in the development process for what has historically been a complex modeling task. We contribute to the information system (IS) discourse with a rapid development framework on the application of generative AI in content online moderation and management of culture in decentralized, pseudonymous communities by providing a sample model suite of industrial-ready generative AI models based on open-access LLMs.

Computers and Society
Degenerative AI?

Steven M. Bellovin

DOI: https://doi.org/10.1109/msec.2024.3385549

IF: 3.105

2024-05-22

IEEE Security & Privacy

Abstract:It is not secret that generative AI, especially in the form of large language models (LLMs), is extremely popular today. One might go so far as to say that it's eaten the world. It may be a bubble, or it may last—though the death of cryptocurrencies has long been predicted, as I write this Bitcoin has just reached an all-time high value against the American dollar—but for now and at least the next few years, generative AI will be with us. As people who care about security and privacy, we need to understand the implications of it: is it good or bad for our field, and if the latter, what should we do about it? Ignoring it is not an option.

computer science, information systems, software engineering
Braess's Paradox of Generative AI

Boaz Taitler,Omer Ben-Porat

DOI: https://doi.org/10.48550/arXiv.2409.05506

2024-09-09

Abstract:ChatGPT has established Generative AI (GenAI) as a significant technological advancement. However, GenAI's intricate relationship with competing platforms and its downstream impact on users remains under-explored. This paper initiates the study of GenAI's long-term social impact resulting from the weakening network effect of human-based platforms like Stack Overflow. First, we study GenAI's revenue-maximization optimization problem. We develop an approximately optimal solution and show that the optimal solution has a non-cyclic structure. Then, we analyze the social impact, showing that GenAI could be socially harmful. Specifically, we present an analog to Braess's paradox in which all users would be better off without GenAI. Finally, we develop necessary and sufficient conditions for a regulator with incomplete information to ensure that GenAI is socially beneficial.

Computer Science and Game Theory
The World of Generative AI: Deepfakes and Large Language Models

Alakananda Mitra,Saraju P. Mohanty,Elias Kougianos

2024-02-07

Abstract:We live in the era of Generative Artificial Intelligence (GenAI). Deepfakes and Large Language Models (LLMs) are two examples of GenAI. Deepfakes, in particular, pose an alarming threat to society as they are capable of spreading misinformation and changing the truth. LLMs are powerful language models that generate general-purpose language. However due to its generative aspect, it can also be a risk for people if used with ill intentions. The ethical use of these technologies is a big concern. This short article tries to find out the interrelationship between them.

Computation and Language
Evolving AI Collectives to Enhance Human Diversity and Enable Self-Regulation

Shiyang Lai,Yujin Potter,Junsol Kim,Richard Zhuang,Dawn Song,James Evans

2024-06-19

Abstract:Large language model behavior is shaped by the language of those with whom they interact. This capacity and their increasing prevalence online portend that they will intentionally or unintentionally "program" one another and form emergent AI subjectivities, relationships, and collectives. Here, we call upon the research community to investigate these "societies" of interacting artificial intelligences to increase their rewards and reduce their risks for human society and the health of online environments. We use a small "community" of models and their evolving outputs to illustrate how such emergent, decentralized AI collectives can spontaneously expand the bounds of human diversity and reduce the risk of toxic, anti-social behavior online. Finally, we discuss opportunities for AI cross-moderation and address ethical issues and design challenges associated with creating and maintaining free-formed AI collectives.

Computation and Language,Computers and Society
The rise of generative artificial intelligence (AI) language models - challenges and opportunities for geographical and environmental education

Chew-Hung Chang,Gillian Kidman

DOI: https://doi.org/10.1080/10382046.2023.2194036

2023-03-29

International Research in Geographical and Environmental Education

Abstract:Much discussion has been surrounding generative AI language models, such as ChatGPT, and the advantages and threats they present to education. As an AI language model, ChatGPT has the potential to transform geography and environmental education. The extensive knowledge base and natural language processing abilities that ChatGPT possesses make it a perfect tool for this purpose; it can actively include students in conversation (as the chat in its name implies) while also providing them with quick feedback. Because of this, students are free to progress through the material at their own pace, using strategies that best suit them individually. Yet, this change has been criticised by others who worry about intellectual property violations and undermining academic integrity. As discussions continue about whether generative AI models are a boon or bane for education, we would like to focus on the problems and potential facing geography and environmental education in particular.
Large Language Models and Generative AI, Oh My!

Peter J. Cobb

DOI: https://doi.org/10.1017/aap.2023.20

2023-09-23

Advances in Archaeological Practice

Abstract:Overview We have all read the headlines heralding, often hyperbolically, the latest advances in text- and image-based Artificial Intelligence (AI). What is perhaps most unique about these developments is that they now make relatively good AI accessible to the average Internet user. These new services respond to human prompts, written in natural language, with generated output that appears to satisfy the prompt. Consequently, they are categorized under the term "generative AI," whether they are generating text, images, or other media. They work by modeling human language statistically, to "learn" patterns from extremely large datasets of human-created content, with those that specifically focus on text therefore called Large Language Models (LLMs). As we have all tried products such as ChatGPT or Midjourney over the past year, we have undoubtedly begun to wonder how and when they might impact our archaeological work. Here, I review the state of this type of AI and the current challenges with using it meaningfully, and I consider its potential for archaeologists.
Exploring the Use of Abusive Generative AI Models on Civitai

Yiluo Wei,Yiming Zhu,Pan Hui,Gareth Tyson

2024-07-21

Abstract:The rise of generative AI is transforming the landscape of digital imagery, and exerting a significant influence on online creative communities. This has led to the emergence of AI-Generated Content (AIGC) social platforms, such as Civitai. These distinctive social platforms allow users to build and share their own generative AI models, thereby enhancing the potential for more diverse artistic expression. Designed in the vein of social networks, they also provide artists with the means to showcase their creations (generated from the models), engage in discussions, and obtain feedback, thus nurturing a sense of community. Yet, this openness also raises concerns about the abuse of such platforms, e.g., using models to disseminate deceptive deepfakes or infringe upon copyrights. To explore this, we conduct the first comprehensive empirical study of an AIGC social platform, focusing on its use for generating abusive content. As an exemplar, we construct a comprehensive dataset covering Civitai, the largest available AIGC social platform. Based on this dataset of 87K models and 2M images, we explore the characteristics of content and discuss strategies for moderation to better govern these platforms.

Social and Information Networks,Artificial Intelligence,Computation and Language
How large language models can reshape collective intelligence

Jason W. Burton,Ezequiel Lopez-Lopez,Shahar Hechtlinger,Zoe Rahwan,Samuel Aeschbach,Michiel A. Bakker,Joshua A. Becker,Aleks Berditchevskaia,Julian Berger,Levin Brinkmann,Lucie Flek,Stefan M. Herzog,Saffron Huang,Sayash Kapoor,Arvind Narayanan,Anne-Marie Nussberger,Taha Yasseri,Pietro Nickl,Abdullah Almaatouq,Ulrike Hahn,Ralf H. J. M. Kurvers,Susan Leavy,Iyad Rahwan,Divya Siddarth,Alice Siu,Anita W. Woolley,Dirk U. Wulff,Ralph Hertwig

DOI: https://doi.org/10.1038/s41562-024-01959-9

IF: 24.252

2024-09-21

Nature Human Behaviour

Abstract:Collective intelligence underpins the success of groups, organizations, markets and societies. Through distributed cognition and coordination, collectives can achieve outcomes that exceed the capabilities of individuals—even experts—resulting in improved accuracy and novel capabilities. Often, collective intelligence is supported by information technology, such as online prediction markets that elicit the 'wisdom of crowds', online forums that structure collective deliberation or digital platforms that crowdsource knowledge from the public. Large language models, however, are transforming how information is aggregated, accessed and transmitted online. Here we focus on the unique opportunities and challenges this transformation poses for collective intelligence. We bring together interdisciplinary perspectives from industry and academia to identify potential benefits, risks, policy-relevant considerations and open research questions, culminating in a call for a closer examination of how large language models affect humans' ability to collectively tackle complex problems.

psychology, experimental,neurosciences,multidisciplinary sciences
Vox Populi, Vox ChatGPT: Large Language Models, Education and Democracy

Niina Zuber,Jan Gogoll

DOI: https://doi.org/10.3390/philosophies9010013

2024-01-11

Philosophies

Abstract:In the era of generative AI and specifically large language models (LLMs), exemplified by ChatGPT, the intersection of artificial intelligence and human reasoning has become a focal point of global attention. Unlike conventional search engines, LLMs go beyond mere information retrieval, entering into the realm of discourse culture. Their outputs mimic well-considered, independent opinions or statements of facts, presenting a pretense of wisdom. This paper explores the potential transformative impact of LLMs on democratic societies. It delves into the concerns regarding the difficulty in distinguishing ChatGPT-generated texts from human output. The discussion emphasizes the essence of authorship, rooted in the unique human capacity for reason—a quality indispensable for democratic discourse and successful collaboration within free societies. Highlighting the potential threats to democracy, this paper presents three arguments: the Substitution argument, the Authenticity argument, and the Facts argument. These arguments highlight the potential risks that are associated with an overreliance on LLMs. The central thesis posits that widespread deployment of LLMs may adversely affect the fabric of a democracy if not comprehended and addressed proactively and properly. In proposing a solution, we advocate for an emphasis on education as a means to mitigate risks. We suggest cultivating thinking skills in children, fostering coherent thought formulation, and distinguishing between machine-generated output and genuine, i.e., human, reasoning. The focus should be on the responsible development and usage of LLMs, with the goal of augmenting human capacities in thinking, deliberating and decision-making rather than substituting them.
Linguistic Landscape of Generative AI Perception: A Global Twitter Analysis Across 14 Languages

Taichi Murayama,Kunihiro Miyazaki,Yasuko Matsubara,Yasushi Sakurai

2024-05-30

Abstract:The advent of generative AI tools has had a profound impact on societies globally, transcending geographical boundaries. Understanding these tools' global reception and utilization is crucial for service providers and policymakers in shaping future policies. Therefore, to unravel the perceptions and engagements of individuals within diverse linguistic communities with regard to generative AI tools, we extensively analyzed over 6.8 million tweets in 14 different languages. Our findings reveal a global trend in the perception of generative AI, accompanied by language-specific nuances. While sentiments toward these tools vary significantly across languages, there is a prevalent positive inclination toward Image tools and a negative one toward Chat tools. Notably, the ban of ChatGPT in Italy led to a sentiment decline and initiated discussions across languages. Furthermore, we established a taxonomy for interactions with chatbots, creating a framework for social analysis underscoring variations in generative AI usage among linguistic communities. We find that the Chinese community predominantly employs chatbots as substitutes for search, while the Italian community tends to present more intricate prompts. Our research provides a robust foundation for further explorations of the social dynamics surrounding generative AI tools and offers invaluable insights for decision-makers in policy, technology, and education.

Computers and Society
Towards Understanding the Interplay of Generative Artificial Intelligence and the Internet

Gonzalo Martínez,Lauren Watson,Pedro Reviriego,José Alberto Hernández,Marc Juarez,Rik Sarkar

DOI: https://doi.org/10.48550/arXiv.2306.06130

IF: 14.4

2023-06-08

Artificial Intelligence

Abstract:The rapid adoption of generative Artificial Intelligence (AI) tools that can generate realistic images or text, such as DALL-E, MidJourney, or ChatGPT, have put the societal impacts of these technologies at the center of public debate. These tools are possible due to the massive amount of data (text and images) that is publicly available through the Internet. At the same time, these generative AI tools become content creators that are already contributing to the data that is available to train future models. Therefore, future versions of generative AI tools will be trained with a mix of human-created and AI-generated content, causing a potential feedback loop between generative AI and public data repositories. This interaction raises many questions: how will future versions of generative AI tools behave when trained on a mixture of real and AI generated data? Will they evolve and improve with the new data sets or on the contrary will they degrade? Will evolution introduce biases or reduce diversity in subsequent generations of generative AI tools? What are the societal implications of the possible degradation of these models? Can we mitigate the effects of this feedback loop? In this document, we explore the effect of this interaction and report some initial results using simple diffusion models trained with various image datasets. Our results show that the quality and diversity of the generated images can degrade over time suggesting that incorporating AI-created data can have undesired effects on future versions of generative models.
AI Meets the Classroom: When Does ChatGPT Harm Learning?

Matthias Lehmann,Philipp B. Cornelius,Fabian J. Sting

2024-08-30

Abstract:In this paper, we study how generative AI and specifically large language models (LLMs) impact learning in coding classes. We show across three studies that LLM usage can have positive and negative effects on learning outcomes. Using observational data from university-level programming courses, we establish such effects in the field. We replicate these findings in subsequent experimental studies, which closely resemble typical learning scenarios, to show causality. We find evidence for two contrasting mechanisms that determine the overall effect of LLM usage on learning. Students who use LLMs as personal tutors by conversing about the topic and asking for explanations benefit from usage. However, learning is impaired for students who excessively rely on LLMs to solve practice exercises for them and thus do not invest sufficient own mental effort. Those who never used LLMs before are particularly prone to such adverse behavior. Students without prior domain knowledge gain more from having access to LLMs. Finally, we show that the self-perceived benefits of using LLMs for learning exceed the actual benefits, potentially resulting in an overestimation of one's own abilities. Overall, our findings show promising potential of LLMs as learning support, however also that students have to be very cautious of possible pitfalls.

Computers and Society,Artificial Intelligence,Human-Computer Interaction,Machine Learning
Generative Monoculture in Large Language Models

Fan Wu,Emily Black,Varun Chandrasekaran

2024-07-02

Abstract:We introduce {\em generative monoculture}, a behavior observed in large language models (LLMs) characterized by a significant narrowing of model output diversity relative to available training data for a given task: for example, generating only positive book reviews for books with a mixed reception. While in some cases, generative monoculture enhances performance (e.g., LLMs more often produce efficient code), the dangers are exacerbated in others (e.g., LLMs refuse to share diverse opinions). As LLMs are increasingly used in high-impact settings such as education and web search, careful maintenance of LLM output diversity is essential to ensure a variety of facts and perspectives are preserved over time. We experimentally demonstrate the prevalence of generative monoculture through analysis of book review and code generation tasks, and find that simple countermeasures such as altering sampling or prompting strategies are insufficient to mitigate the behavior. Moreover, our results suggest that the root causes of generative monoculture are likely embedded within the LLM's alignment processes, suggesting a need for developing fine-tuning paradigms that preserve or promote diversity.

Computation and Language,Artificial Intelligence
From ChatGPT, DALL-E 3 to Sora: How has Generative AI Changed Digital Humanities Research and Services?

Jiangfeng Liu,Ziyi Wang,Jing Xie,Lei Pei

2024-04-29

Abstract:Generative large-scale language models create the fifth paradigm of scientific research, organically combine data science and computational intelligence, transform the research paradigm of natural language processing and multimodal information processing, promote the new trend of AI-enabled social science research, and provide new ideas for digital humanities research and application. This article profoundly explores the application of large-scale language models in digital humanities research, revealing their significant potential in ancient book protection, intelligent processing, and academic innovation. The article first outlines the importance of ancient book resources and the necessity of digital preservation, followed by a detailed introduction to developing large-scale language models, such as ChatGPT, and their applications in document management, content understanding, and cross-cultural research. Through specific cases, the article demonstrates how AI can assist in the organization, classification, and content generation of ancient books. Then, it explores the prospects of AI applications in artistic innovation and cultural heritage preservation. Finally, the article explores the challenges and opportunities in the interaction of technology, information, and society in the digital humanities triggered by AI technologies.

Digital Libraries,Artificial Intelligence,Computation and Language,Computers and Society
Generative AI: An Existential Threat to Human Content Creators?

Jiasun Li

DOI: https://doi.org/10.2139/ssrn.4666860

2023-01-01

SSRN Electronic Journal

Abstract:With the phenomenal rise of generative AI models (e.g., Large Language Models such as GPT or text-to-image models such as Stable Diffusion), there are increasing concerns about human content creators' future. Specifically, as generative models further advance, would they eventually take over the jobs of all human content creators? We argue that the answer is no, even if generative AI models can perform at their theoretical limit one day. Our theory has a close analogy to the familiar insight in financial economics on the impossibility of a perfectly informationally efficient market as in Grossman and Stiglitz (1980): If generative AI models can provide all the content that human needs at low variable costs, then there is no incentive for human content creators to spend resources on content creation as they cannot profit from it. But if no human creates new content, then generative AI models can only rehash what they already know and be unable to provide up-to-date answers. This creates a paradox.
Generative AI enhances individual creativity but reduces the collective diversity of novel content

Anil R. Doshi,Oliver P. Hauser

DOI: https://doi.org/10.1126/sciadv.adn5290

IF: 13.6

2024-07-13

Science Advances

Abstract:Creativity is core to being human. Generative artificial intelligence (AI)—including powerful large language models (LLMs)—holds promise for humans to be more creative by offering new ideas, or less creative by anchoring on generative AI ideas. We study the causal impact of generative AI ideas on the production of short stories in an online experiment where some writers obtained story ideas from an LLM. We find that access to generative AI ideas causes stories to be evaluated as more creative, better written, and more enjoyable, especially among less creative writers. However, generative AI–enabled stories are more similar to each other than stories by humans alone. These results point to an increase in individual creativity at the risk of losing collective novelty. This dynamic resembles a social dilemma: With generative AI, writers are individually better off, but collectively a narrower scope of novel content is produced. Our results have implications for researchers, policy-makers, and practitioners interested in bolstering creativity.

multidisciplinary sciences

Generative AI Degrades Online Communities

The Consequences of Generative AI for UGC and Online Community Engagement

AI models collapse when trained on recursively generated data

Generative Language Models Exhibit Social Identity Biases

Can AI Moderate Online Communities?

Degenerative AI?

Braess's Paradox of Generative AI

The World of Generative AI: Deepfakes and Large Language Models

Evolving AI Collectives to Enhance Human Diversity and Enable Self-Regulation

The rise of generative artificial intelligence (AI) language models - challenges and opportunities for geographical and environmental education

Large Language Models and Generative AI, Oh My!

Exploring the Use of Abusive Generative AI Models on Civitai

How large language models can reshape collective intelligence

Vox Populi, Vox ChatGPT: Large Language Models, Education and Democracy

Linguistic Landscape of Generative AI Perception: A Global Twitter Analysis Across 14 Languages

Towards Understanding the Interplay of Generative Artificial Intelligence and the Internet

AI Meets the Classroom: When Does ChatGPT Harm Learning?

Generative Monoculture in Large Language Models

From ChatGPT, DALL-E 3 to Sora: How has Generative AI Changed Digital Humanities Research and Services?

Generative AI: An Existential Threat to Human Content Creators?

Generative AI enhances individual creativity but reduces the collective diversity of novel content