The Impact of Large Language Models in Academia: from Writing to Speaking

Mingmeng Geng,Caixi Chen,Yanru Wu,Dongping Chen,Yao Wan,Pan Zhou
2024-10-23
Abstract:Large language models (LLMs) are increasingly impacting human society, particularly in textual information. Based on more than 30,000 papers and 1,000 presentations from machine learning conferences, we examined and compared the words used in writing and speaking, representing the first large-scale study of how LLMs influence the two main modes of verbal communication and expression within the same group of people. Our empirical results show that LLM-style words such as "significant" have been used more frequently in abstracts and oral presentations. The impact on speaking is beginning to emerge and is likely to grow in the future, calling attention to the implicit influence and ripple effect of LLMs on human society.
Computation and Language,Artificial Intelligence,Computers and Society,Digital Libraries,Machine Learning
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the impact of large language models (LLMs) on academic writing and oral expression. Specifically, by analyzing more than 30,000 paper abstracts and 1,000 conference speeches, the authors explored how LLMs affect the use of words and oral expressions in academic communication. This is the first large - scale study on how the same group is affected by LLMs in two major language communication methods, writing and speaking. ### Main research questions: 1. **The impact of LLMs on writing**: Research whether and how LLMs have changed the vocabulary and expressions used in academic papers. 2. **The impact of LLMs on speaking**: Explore whether and how LLMs have affected the vocabulary selection and expressions in academic conference speeches. 3. **Comparison between writing and speaking**: Compare the similarities and differences in how the same group is affected by LLMs in writing and speaking, especially the changes in the frequency of use of certain specific words. ### Research methods: - **Data collection**: A large number of paper abstracts and speech videos were collected from machine learning conferences (such as ICML, ICLR, NeurIPS). - **Word frequency analysis**: By analyzing the frequency of occurrence of specific words in abstracts and speeches, the impact of LLMs was evaluated. - **LLM simulation**: GPT - 3.5 was used to modify the abstracts to simulate the processing effect of LLMs, and the changes in word frequency before and after the modification were compared. - **Impact estimation**: The overall impact of LLMs on the text was estimated by statistical methods. ### Key findings: - **Word frequency changes**: The use frequency of certain specific words (such as "significant", "crucial", etc.) has increased significantly in paper abstracts and speeches after 2022. - **Differences between writing and speaking**: Although the impact of LLMs on writing is more obvious, its impact on speaking has also begun to emerge and may further increase in the future. - **Implicit impact**: Even if people do not directly use LLMs to generate content, their language expressions are also affected by being exposed to the content generated by these models. ### Conclusion: This study is the first to systematically analyze the impact of LLMs in the writing and speaking of the same group, revealing the implicit impact of LLMs on academic communication and its potential long - term effects. Although currently this impact is not as obvious in speaking as in writing, it may be more significant in the future. This finding is of great significance for understanding the role of LLMs in human society.