Are Preprints a Threat to the Credibility and Quality of Artificial Intelligence Literature in the ChatGPT Era? A Scoping Review and Qualitative Study
Michael Agyemang Adarkwah A. Y. M. Atiquil Islam Käthe Schneider Rose Luckin Michael Thomas Jonathan Michael Spector a Institute for Education and Culture,Friedrich-Schiller-Universität Jena,Jena,Germanyb Department of Education Information Technology,East China Normal University,Shanghai,Chinac School of Teacher Education,Jiangsu University,Jiangsu,Chinad Institute of Education and Culture,University of Jena,Germanye UCL Knowledge Lab,Institute of Education,University College London,London,UKf School of Education,Liverpool John Moores University,UKg Department of Learning Technologies,College of Information,University of North Texas,Denton,USAMichael Agyemang Adarkwah is a Research Associate at the Institute for Education and Culture,at the Chair of Adult Education,Friedrich Schiller University Jena,Germany. His habilitation is on the application and implication of learning technologies in adult education in this AI era with a focus on challenges,policy,and best practices.A. Y. M. Atiquil Islam is an Associate Professor at the Department of Education Information Technology of the East China Normal University. His research interests are in the arena of Assessment of Educational Technology,Artificial Intelligence and ICT in Higher Education. He is an Editorial Board Member of the British Journal of Educational Technology.Käthe Schneider is a Professor and Chair holder at the Institute for Education and Culture,at the Chair of Adult Education,Friedrich Schiller University Jena,Germany. Her research interests in the area of Adult Education.Rose Luckin is a Professor and an expert in AI for education at University College London. She founded Educate Ventures Research to advance edtech and AI for learning. Professor Luckin is renowned for her research on educational technology and AI,and for writing accessible books like Machine Learning and Human Intelligence.Michael Thomas is a Professor in Education and Social Justice and Director of the Centre for Educational Research in the School of Education at the Liverpool John Moores University. His primary research focus is digital education and social justice across a range of sectors from higher education to schools.Jonathan Michael Spector is a Regents Professor at the Department of Learning Technologies at the University of North Texas. His recent research is in the areas of intelligent support for instructional design and technology integration in education. He was the editor-in-chief of Educational Technology Research & Development for the last 15 years.
DOI: https://doi.org/10.1080/10447318.2024.2364140
IF: 4.92
2024-06-19
International Journal of Human-Computer Interaction
Abstract:ChatGPT, as the pioneer of advanced generative AI tools, has triggered scholarly discussions about the potential use of such AI technologies in interdisciplinary fields. With a focus on the surge of AI-related preprints since the introduction of ChatGPT by OpenAI, the study investigated what the surge implies for AI literature, particularly in terms of credibility and quality. A scoping review was initially conducted to study the characteristics of the AI-related preprints in the Web of Science (WoS) database and also in five (5) preprint platforms (ArXiv, MedRxiv, SocArxiv, SSRN, and Research Square). The publication date range was set at January 01, 2023 to September 08, 2023. This was followed up by an interpretive phenomenological analysis (IPA) of the perceptions of experts in the AI field about the preprints. Employing a scoping review of AI-related preprints across six databases and a qualitative analysis of 15 AI experts' opinions, our study reveals concerns about the research accuracy, quality, and credibility of preprints, and advocates for a robust evaluation and high-quality assurance process to promote open science objectives during their dissemination. Specifically, 45,918 AI-related preprints were found in the six preprint databases or repositories across different fields. The nine themes from the IPA showed that preprints can be of value. However, experts advocated for the safe and responsible use of AI-related preprints, involving such tenets as maintaining ethical integrity and high-quality work on the part of authors and establishing sound AI-content guidelines from publishers and editors. Future studies are recommended to investigate the impact of preprints on decision-making processes in educational research and practice.
computer science, cybernetics,ergonomics