Guidelines for academics aim to lessen ethical pitfalls in generative-AI use

Liam Drew

DOI: https://doi.org/10.1038/d41586-024-01543-1

IF: 64.8

2024-05-24

Nature

Abstract:Researchers warn against normalizing the use of AI without safeguarding against risks. Researchers warn against normalizing the use of AI without safeguarding against risks.

multidisciplinary sciences

What problem does this paper attempt to address?

Social Risks in the Era of Generative AI

Xiaozhong Liu,Yu‐Ru Lin,Zhuoren Jiang,Qunfang Wu

DOI: https://doi.org/10.1002/pra2.1103

2024-01-01

Proceedings of the Association for Information Science and Technology

Abstract:ABSTRACTGenerative AI (GAI) technologies have demonstrated human‐level performance on a vast spectrum of tasks. However, recent studies have also delved into the potential threats and vulnerabilities posed by GAI, particularly as they become increasingly prevalent in sensitive domains such as elections and education. Their use in politics raises concerns about manipulation and misinformation. Further exploration is imperative to comprehend the social risks associated with GAI across diverse societal contexts. In this panel, we aim to dissect the impact and risks posed by GAI on our social fabric, examining both technological and societal perspectives. Additionally, we will present our latest investigations, including the manipulation of ideologies using large language models (LLMs), the potential risk of AI self‐consciousness, the application of Explainable AI (XAI) to identify patterns of misinformation and mitigate their dissemination, as well as the influence of GAI on the quality of public discourse. These insights will serve as catalysts for stimulating discussions among the audience on this crucial subject matter, and contribute to fostering a deeper understanding of the importance of responsible development and deployment of GAI technologies.
The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool

David B. Resnik,Mohammad Hosseini

DOI: https://doi.org/10.1007/s43681-024-00493-8

2024-05-27

AI and Ethics

Abstract:Abstract Using artificial intelligence (AI) in research offers many important benefits for science and society but also creates novel and complex ethical issues. While these ethical issues do not necessitate changing established ethical norms of science, they require the scientific community to develop new guidance for the appropriate use of AI. In this article, we briefly introduce AI and explain how it can be used in research, examine some of the ethical issues raised when using it, and offer nine recommendations for responsible use, including: (1) Researchers are responsible for identifying, describing, reducing, and controlling AI-related biases and random errors; (2) Researchers should disclose, describe, and explain their use of AI in research, including its limitations, in language that can be understood by non-experts; (3) Researchers should engage with impacted communities, populations, and other stakeholders concerning the use of AI in research to obtain their advice and assistance and address their interests and concerns, such as issues related to bias; (4) Researchers who use synthetic data should (a) indicate which parts of the data are synthetic; (b) clearly label the synthetic data; (c) describe how the data were generated; and (d) explain how and why the data were used; (5) AI systems should not be named as authors, inventors, or copyright holders but their contributions to research should be disclosed and described; (6) Education and mentoring in responsible conduct of research should include discussion of ethical use of AI.
A University Framework for the Responsible use of Generative AI in Research

Shannon Smith,Melissa Tate,Keri Freeman,Anne Walsh,Brian Ballsun-Stanton,Mark Hooper,Murray Lane

2024-04-30

Abstract:Generative Artificial Intelligence (generative AI) poses both opportunities and risks for the integrity of research. Universities must guide researchers in using generative AI responsibly, and in navigating a complex regulatory landscape subject to rapid change. By drawing on the experiences of two Australian universities, we propose a framework to help institutions promote and facilitate the responsible use of generative AI. We provide guidance to help distil the diverse regulatory environment into a principles-based position statement. Further, we explain how a position statement can then serve as a foundation for initiatives in training, communications, infrastructure, and process change. Despite the growing body of literature about AI's impact on academic integrity for undergraduate students, there has been comparatively little attention on the impacts of generative AI for research integrity, and the vital role of institutions in helping to address those challenges. This paper underscores the urgency for research institutions to take action in this area and suggests a practical and adaptable framework for so doing.

Computers and Society,Artificial Intelligence
The global landscape of academic guidelines for generative AI and Large Language Models

Junfeng Jiao,Saleh Afroogh,Kevin Chen,David Atkinson,Amit Dhurandhar

2024-06-28

Abstract:The integration of Generative Artificial Intelligence (GAI) and Large Language Models (LLMs) in academia has spurred a global discourse on their potential pedagogical benefits and ethical considerations. Positive reactions highlight some potential, such as collaborative creativity, increased access to education, and empowerment of trainers and trainees. However, negative reactions raise concerns about ethical complexities, balancing innovation and academic integrity, unequal access, and misinformation risks. Through a systematic survey and text-mining-based analysis of global and national directives, insights from independent research, and eighty university-level guidelines, this study provides a nuanced understanding of the opportunities and challenges posed by GAI and LLMs in education. It emphasizes the importance of balanced approaches that harness the benefits of these technologies while addressing ethical considerations and ensuring equitable access and educational outcomes. The paper concludes with recommendations for fostering responsible innovation and ethical practices to guide the integration of GAI and LLMs in academia.

Computers and Society,Artificial Intelligence,Computation and Language
Editors' Statement on the Responsible Use of Generative AI Technologies in Scholarly Journal Publishing

Gregory E Kaebnick,David Christopher Magnus,Audiey Kao,Mohammad Hosseini,David Resnik,Veljko Dubljević,Christy Rentmeester,Bert Gordijn,Mark J Cherry,Karen J Maschke,John McMillan,Lisa M Rasmussen,Laura Haupt,Udo Schüklenk,Ruth Chadwick,Debora Diniz

DOI: https://doi.org/10.1002/eahr.500182

Abstract:Generative artificial intelligence (AI) has the potential to transform many aspects of scholarly publishing. Authors, peer reviewers, and editors might use AI in a variety of ways, and those uses might augment their existing work or might instead be intended to replace it. We are editors of bioethics and humanities journals who have been contemplating the implications of this ongoing transformation. We believe that generative AI may pose a threat to the goals that animate our work but could also be valuable for achieving those goals. In the interests of fostering a wider conversation about how generative AI may be used, we have developed a preliminary set of recommendations for its use in scholarly publishing. We hope that the recommendations and rationales set out here will help the scholarly community navigate toward a deeper understanding of the strengths, limits, and challenges of AI for responsible scholarly work.
Ethical Considerations for AI Researchers

Kyle Dent

DOI: https://doi.org/10.48550/arXiv.2006.07558

2020-06-13

Computers and Society

Abstract:Use of artificial intelligence is growing and expanding into applications that impact people's lives. People trust their technology without really understanding it or its limitations. There is the potential for harm and we are already seeing examples of that in the world. AI researchers have an obligation to consider the impact of intelligent applications they work on. While the ethics of AI is not clear-cut, there are guidelines we can consider to minimize the harm we might introduce.
Beyond principlism: Practical strategies for ethical AI use in research practices

Zhicheng Lin

DOI: https://doi.org/10.1007/s43681-024-00585-5

2024-10-04

Abstract:The rapid adoption of generative artificial intelligence (AI) in scientific research, particularly large language models (LLMs), has outpaced the development of ethical guidelines, leading to a Triple-Too problem: too many high-level ethical initiatives, too abstract principles lacking contextual and practical relevance, and too much focus on restrictions and risks over benefits and utilities. Existing approaches, including principlism (reliance on abstract ethical principles), formalism (rigid application of rules), and technical solutionism (overemphasis on technological fixes), offer little practical guidance for addressing ethical challenges of AI in scientific research practices. To bridge the gap between abstract principles and day-to-day research practices, a user-centered, realism-inspired approach is proposed here. It outlines five specific goals for ethical AI use: 1) understanding model training and output, including bias mitigation strategies; 2) respecting privacy, confidentiality, and copyright; 3) avoiding plagiarism and policy violations; 4) applying AI beneficially compared to alternatives; and 5) using AI transparently and reproducibly. Each goal is accompanied by actionable strategies and realistic cases of misuse and corrective measures. I argue that ethical AI application requires evaluating its utility against existing alternatives rather than isolated performance metrics. Additionally, I propose documentation guidelines to enhance transparency and reproducibility in AI-assisted research. Moving forward, we need targeted professional development, training programs, and balanced enforcement mechanisms to promote responsible AI use while fostering innovation. By refining these ethical guidelines and adapting them to emerging AI capabilities, we can accelerate scientific progress without compromising research integrity.

Computers and Society,Artificial Intelligence
Appraising Regulatory Framework Towards Artificial General Intelligence (AGI) under Digital Humanism

Le Cheng,Xuan Gong

DOI: https://doi.org/10.1515/ijdlg-2024-0015

2024-01-01

Abstract:Abstract The explosive advancement of contemporary artificial intelligence (AI) technologies, typified by ChatGPT, is steering humans towards an uncontrollable trajectory to artificial general intelligence (AGI). Against the backdrop of a series of transformative breakthroughs, big tech companies such as OpenAI and Google have initiated an “AGI race” on a supranational level. As technological power becomes increasingly absolute, structural challenges may erupt with an unprecedented velocity, potentially resulting in disorderly expansion and even malignant development of AI technologies. To preserve the dignity and safety of human-beings in a brand-new AGI epoch, it is imperative to implement regulatory guidelines to limit the applications of AGI within the confines of human ethics and rules to further counteract the potential downsides. To promote the benevolent evolution of AGI, the principles of Humanism should be underscored and the connotation of Digital Humanism should be further enriched. Correspondingly, the current regulatory paradigm for generative AI may also be overhauled under the tenet of Digital Humanism to adapt to the quantum leaps and subversive shifts produced by AGI in the future. Positioned at the nexus of legal studies, computer science, and moral philosophy, this study therefore charts a course for a synthetic regulation framework of AGI under Digital Humanism.
Nothing Is Harder to Resist Than the Temptation of AI

Andrew Park,Jan Kietzmann,Jayson Killoran,Yuanyuan Cui,Patrick van Esch,Amir Dabirian

DOI: https://doi.org/10.1109/mitp.2023.3340529

2024-02-03

IT Professional

Abstract:The use of generative AI has become increasingly prevalent in the business world. With the ability to create original content and automate certain tasks, businesses have been quick to adopt this technology. However, as with any emerging technology, there are potential pitfalls to be aware of. This article aims to review the current state of generative AI in business and highlight some of the potential risks associated with its use. Specifically, we examine issues such as pausing giant AI experiments, misinformation, data accuracy, process automation, shift of power, control of civilization, organizational security, and the potential for AI-generated content to deceive individuals. By bringing these concerns to the forefront, we hope to encourage a more thoughtful and cautious approach to the use of generative AI in business.

computer science, information systems,telecommunications, software engineering
Academic publisher guidelines on AI usage: A ChatGPT supported thematic analysis

Mike Perkins,Jasper Roe

DOI: https://doi.org/10.12688/f1000research.142411.2

2024-01-16

Abstract:Background: As Artificial Intelligence (AI) technologies such as Generative AI (GenAI) have become more common in academic settings, it is necessary to examine how these tools interact with issues of authorship, academic integrity, and research methodologies. The current landscape lacks cohesive policies and guidelines for regulating AI's role in academic research which has prompted discussions among publishers, authors, and institutions. Methods: This study employs inductive thematic analysis to explore publisher policies regarding AI-assisted authorship and academic work. Our methods involved a two-fold analysis using both AI-assisted and traditional unassisted techniques to examine the available policies from leading academic publishers and other publishing or academic entities. The framework was designed to offer multiple perspectives, harnessing the strengths of AI for pattern recognition while leveraging human expertise for nuanced interpretation. The results of these two analyses are combined to form the final themes. Results: Our findings indicate six overall themes, three of which were independently identified in both the AI-assisted and unassisted, manual analysis using common software tools. A broad consensus appears among publishers that human authorship remains paramount and that the use of GenAI tools is permissible but must be disclosed. However, GenAI tools are increasingly acknowledged for their supportive roles, including text generation and data analysis. The study also discusses the inherent limitations and biases of AI-assisted analysis, necessitating rigorous scrutiny by authors, reviewers, and editors. Conclusions: There is a growing recognition of AI's role as a valuable auxiliary tool in academic research, but one that comes with caveats pertaining to integrity, accountability, and interpretive limitations. This study used a novel analysis supported by GenAI tools to identify themes emerging in the policy landscape, underscoring the need for an informed, flexible approach to policy formulation that can adapt to the rapidly evolving landscape of AI technologies.
AI and ethics: Investigating the first policy responses of higher education institutions to the challenge of generative AI

Attila Dabis,Csaba Csáki

DOI: https://doi.org/10.1057/s41599-024-03526-z

2024-08-07

Humanities and Social Sciences Communications

Abstract:This article addresses the ethical challenges posed by generative artificial intelligence (AI) tools in higher education and explores the first responses of universities to these challenges globally. Drawing on five key international documents from the UN, EU, and OECD, the study used content analysis to identify key ethical dimensions related to the use of generative AI in academia, such as accountability, human oversight, transparency, or inclusiveness. Empirical evidence was compiled from 30 leading universities ranked among the top 500 in the Shanghai Ranking list from May to July 2023, covering those institutions that already had publicly available responses to these dimensions in the form of policy documents or guidelines. The paper identifies the central ethical imperative that student assignments must reflect individual knowledge acquired during their education, with human individuals retaining moral and legal responsibility for AI-related wrongdoings. This top-down requirement aligns with a bottom-up approach, allowing instructors flexibility in determining how they utilize generative AI especially large language models in their own courses. Regarding human oversight, the typical response identified by the study involves a blend of preventive measures (e.g., course assessment modifications) and soft, dialogue-based sanctioning procedures. The challenge of transparency induced the good practice of clear communication of AI use in course syllabi in the first university responses examined by this study.

English Else
A plea for caution and guidance about using AI in genomics

Mohammad Hosseini,Christopher R. Donohue

DOI: https://doi.org/10.1038/s42256-024-00947-y

IF: 23.8

2024-12-01

Nature Machine Intelligence

Abstract:The incorporation of artificial intelligence (AI) into genetics and genomics research can enable research that would have been otherwise impossible. However, these benefits must be considered together with the potential risks to humans, other sentient beings, and the environment. Genetic and genomic advances require much trial and error to succeed; this is ethically fraught when the consequences are unknown and the moral status of created or modified tissues or organisms is unclear, or possibly comparable to conscious beings. We argue that it is urgent to expand the ethical discourse on the use of AI in genomics research and to develop appropriate guidance. Genetics and genomics research has enabled us to answer questions about historical events and keeps adding considerable depth to what we know about the world and ourselves. For example, it has determined when and how certain animals and plants were domesticated, how diseases spread, how viruses mutate, and thanks to CRISPR, has led to remarkable advances in the understanding and treatment of rare genetic conditions. However, these advances also intensify long-standing tensions between biomedical innovations and research ethics norms, especially with new AI developments.

computer science, artificial intelligence, interdisciplinary applications
Generative AI in Higher Education: Seeing ChatGPT Through Universities' Policies, Resources, and Guidelines

Hui Wang,Anh Dang,Zihao Wu,Son Mac

2024-07-12

Abstract:The advancements in Generative Artificial Intelligence (GenAI) provide opportunities to enrich educational experiences, but also raise concerns about academic integrity. Many educators have expressed anxiety and hesitation in integrating GenAI in their teaching practices, and are in needs of recommendations and guidance from their institutions that can support them to incorporate GenAI in their classrooms effectively. In order to respond to higher educators' needs, this study aims to explore how universities and educators respond and adapt to the development of GenAI in their academic contexts by analyzing academic policies and guidelines established by top-ranked U.S. universities regarding the use of GenAI, especially ChatGPT. Data sources include academic policies, statements, guidelines, and relevant resources provided by the top 100 universities in the U.S. Results show that the majority of these universities adopt an open but cautious approach towards GenAI. Primary concerns lie in ethical usage, accuracy, and data privacy. Most universities actively respond and provide diverse types of resources, such as syllabus templates, workshops, shared articles, and one-on-one consultations focusing on a range of topics: general technical introduction, ethical concerns, pedagogical applications, preventive strategies, data privacy, limitations, and detective tools. The findings provide four practical pedagogical implications for educators in teaching practices: accept its presence, align its use with learning objectives, evolve curriculum to prevent misuse, and adopt multifaceted evaluation strategies rather than relying on AI detectors. Two recommendations are suggested for educators in policy making: establish discipline-specific policies and guidelines, and manage sensitive information carefully.

Computation and Language,Computers and Society
Using Generative AI for Literature Searches and Scholarly Writing: Is the Integrity of the Scientific Discourse in Jeopardy?

Paul G. Schmidt,Amnon J. Meir

DOI: https://doi.org/10.48550/arXiv.2311.06981

2023-10-08

Abstract:Ever since the public release of ChatGPT in November 2022, serious concerns have been raised about the impact and potentially dire consequences of the increasingly widespread use of generative AI tools for purposes of scientific writing and publishing. We document the ongoing discussion in the science community with a review of news articles, editorials, and position statements by major scientific publishers; discuss the potential pitfalls of using generative AI tools such as ChatGPT as aids in scholarly writing; furnish evidence for the proposition that AI-induced contamination of the scientific literature is not only a threat, but already a reality; and call upon leaders in our field to develop policies and guidelines to stem the spread of such contamination. Closing on a positive note, we provide a brief overview of potentially useful capabilities and sensible applications of ChatGPT and similar AI tools for purposes of scholarly writing.

Computers and Society,General Mathematics,History and Overview
Tools such as ChatGPT threaten transparent science; here are our ground rules for their use

DOI: https://doi.org/10.1038/d41586-023-00191-1

IF: 64.8

2023-01-25

Nature

Abstract:As researchers dive into the brave new world of advanced AI chatbots, publishers need to acknowledge their legitimate uses and lay down clear guidelines to avoid abuse. As researchers dive into the brave new world of advanced AI chatbots, publishers need to acknowledge their legitimate uses and lay down clear guidelines to avoid abuse.

multidisciplinary sciences
Higher education crisis: Academic misconduct with generative AI

NaYoung Song

DOI: https://doi.org/10.1111/1468-5973.12532

2024-01-09

Journal of Contingencies and Crisis Management

Abstract:Abstract Higher educational institutions (HEIs) are facing a significant challenge in maintaining academic integrity due to the technological integration of generative artificial intelligence (AI). The widespread use of AI tools by college students has resulted in an increase in plagiarism and cheating, highlighting the need for effective implementation of this technology. However, there is a lack of research on the best practices for using AI in academic settings. HEIs must take responsibility for addressing these issues, as the majority of institutions do not have formal guidelines for AI use, leading to confusion among students and instructors. To combat academic misconduct, HEIs should establish clear objectives and policies for the equitable, inclusive, and ethical use of AI. Improving AI literacy among students and faculty is crucial, as it ensures that everyone has equal access to technology, preventing a digital divide. Moreover, proactive education on the ethical use of AI is vital for HEIs to prepare students for the AI‐driven future of education and maintain academic integrity.

management
'Without these tools, I'd be lost': how generative AI aids in accessibility

Amanda Heidt

DOI: https://doi.org/10.1038/d41586-024-01003-w

IF: 64.8

2024-04-09

Nature

Abstract:A rush to place barriers around the use of artificial intelligence in academia could disproportionately affect those who stand to benefit most. A rush to place barriers around the use of artificial intelligence in academia could disproportionately affect those who stand to benefit most.

multidisciplinary sciences
Danish university policies on generative AI

Olivier Driessens,Magda Pischetola

DOI: https://doi.org/10.7146/mk.v40i76.143595

2024-08-30

Abstract:The sudden and meteoric rise of generative Artificial Intelligence (genAI) has raised fundamental concerns for universities. Using Bacchi’s methodology on ‘problematisation’, we analyse which concerns Danish universities have addressed through their policies and guidelines. We identify three key problematisations: assessment integrity, legality of data and veracity. While each of these problematisations involves specific limitations, together they also strongly emphasise symbolic and epistemological issues and consequently mostly ignore the materiality of genAI, for example, in terms of labour and energy use. Drawing on critical AI studies, this article argues that universities should also consider the huge planetarycosts that (gen)AI poses as well as the full range of AI’s exploitative business models and practices. Universities should integrate these considerations into both their decision-making on (not) using certain technologies and their policies and guidelines for research and teaching, just as sustainability is already a criterion in their travel or investment policies today.
Why and how to embrace AI such as ChatGPT in your academic life

Zhicheng Lin

DOI: https://doi.org/10.1098/rsos.230658

IF: 3.5

2023-08-24

Royal Society Open Science

Abstract:Generative artificial intelligence (AI), including large language models (LLMs), is poised to transform scientific research, enabling researchers to elevate their research productivity. This article presents a how-to guide for employing LLMs in academic settings, focusing on their unique strengths, constraints and implications through the lens of philosophy of science and epistemology. Using ChatGPT as a case study, I identify and elaborate on three attributes contributing to its effectiveness—intelligence, versatility and collaboration—accompanied by tips on crafting effective prompts, practical use cases and a living resource online (https://osf.io/8vpwu/). Next, I evaluate the limitations of generative AI and its implications for ethical use, equality and education. Regarding ethical and responsible use, I argue from technical and epistemic standpoints that there is no need to restrict the scope or nature of AI assistance, provided that its use is transparently disclosed. A pressing challenge, however, lies in detecting fake research, which can be mitigated by embracing open science practices, such as transparent peer review and sharing data, code and materials. Addressing equality, I contend that while generative AI may promote equality for some, it may simultaneously exacerbate disparities for others—an issue with potentially significant yet unclear ramifications as it unfolds. Lastly, I consider the implications for education, advocating for active engagement with LLMs and cultivating students' critical thinking and analytical skills. The how-to guide seeks to empower researchers with the knowledge and resources necessary to effectively harness generative AI while navigating the complex ethical dilemmas intrinsic to its application.

multidisciplinary sciences
Suspected Undeclared Use of Artificial Intelligence in the Academic Literature: An Analysis of the Academ-AI Dataset

Alex Glynn

2024-11-21

Abstract:Since generative artificial intelligence (AI) tools such as OpenAI's ChatGPT became widely available, researchers have used them in the writing process. The consensus of the academic publishing community is that such usage must be declared in the published article. Academ-AI documents examples of suspected undeclared AI usage in the academic literature, discernible primarily due to the appearance in research papers of idiosyncratic verbiage characteristic of large language model (LLM)-based chatbots. This analysis of the first 500 examples collected reveals that the problem is widespread, penetrating the journals and conference proceedings of highly respected publishers. Undeclared AI seems to appear in journals with higher citation metrics and higher article processing charges (APCs), precisely those outlets that should theoretically have the resources and expertise to avoid such oversights. An extremely small minority of cases are corrected post publication, and the corrections are often insufficient to rectify the problem. The 500 examples analyzed here likely represent a small fraction of the undeclared AI present in the academic literature, much of which may be undetectable. Publishers must enforce their policies against undeclared AI usage in cases that are detectable; this is the best defense currently available to the academic publishing community against the proliferation of undisclosed AI.

Digital Libraries,Artificial Intelligence,Computers and Society

Guidelines for academics aim to lessen ethical pitfalls in generative-AI use

Social Risks in the Era of Generative AI

The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool

A University Framework for the Responsible use of Generative AI in Research

The global landscape of academic guidelines for generative AI and Large Language Models

Editors' Statement on the Responsible Use of Generative AI Technologies in Scholarly Journal Publishing

Ethical Considerations for AI Researchers

Beyond principlism: Practical strategies for ethical AI use in research practices

Appraising Regulatory Framework Towards Artificial General Intelligence (AGI) under Digital Humanism

Nothing Is Harder to Resist Than the Temptation of AI

Academic publisher guidelines on AI usage: A ChatGPT supported thematic analysis

AI and ethics: Investigating the first policy responses of higher education institutions to the challenge of generative AI

A plea for caution and guidance about using AI in genomics

Generative AI in Higher Education: Seeing ChatGPT Through Universities' Policies, Resources, and Guidelines

Using Generative AI for Literature Searches and Scholarly Writing: Is the Integrity of the Scientific Discourse in Jeopardy?

Tools such as ChatGPT threaten transparent science; here are our ground rules for their use

Higher education crisis: Academic misconduct with generative AI

'Without these tools, I'd be lost': how generative AI aids in accessibility

Danish university policies on generative AI

Why and how to embrace AI such as ChatGPT in your academic life

Suspected Undeclared Use of Artificial Intelligence in the Academic Literature: An Analysis of the Academ-AI Dataset