Abstract:1565 Background: Generative AI models such as OpenAI’s ChatGPT have been broadly utilized throughout the medical literature. Previous studies have found that AI can generate scientific abstracts which can be difficult to distinguish from the work of human authors. There is a pressing need to characterize utilization of AI in scientific writing to guide policy. Methods: In collaboration with ASCO's Center for Research and Analytics, we extracted text from all scientific abstracts submitted to ASCO 2021 – 2023 Annual Meetings. Likelihood of AI content was evaluated by four AI detectors: GPTZero, Originality.ai (OGAI), Sapling, and Kashyap's AI Content Detector (AICD). Each detector produces a numeric likelihood of AI content. Predictions were dichotomized for uniform analysis across detectors: those in the top 5% for a given detector were considered as having high likelihood of AI content. Logistic regression was used to compute odds ratio (OR) for AI-generated content with respect to submission year and abstract characteristics. Predictions were also assessed for 10 human-written abstracts as negative controls, and 10 produced by OpenAI’s GPT-3 and GPT-4 models as positive controls. Results: 15,553 abstracts met inclusion criteria. 5,179 (33%) were published online only, and 5,327 (34%) referenced registered clinical trials. Across all detectors, abstracts submitted in 2023 were significantly more likely to contain AI content than those in 2021 (OR range 1.3 - 1.7). In abstracts from 2023, AI content score was associated with online only publication, lack of clinical trial number, and abstract track (Table). None of the 10 negative control human written abstracts were identified as AI generated, whereas 100%, 95%, 90%, and 30% of the GPT-3/4 generated abstracts were classified as AI generated by Sapling, GPTZero, OGAI, and AICD respectively using the 5% threshold. Additional results will be presented. Conclusions: AI content detectors uniformly suggest a higher likelihood of AI content generation for abstracts submitted in 2023. Predicted AI content is associated with triage of abstracts to online only presentation, suggesting predicted AI content is associated with lower perceived abstract quality. Further work is needed to understand the accuracy of AI detectors and utility in the abstract review process. [Table: see text]

A Rapid Investigation of Artificial Intelligence Generated Content Footprints in Scholarly Publications

Suspected Undeclared Use of Artificial Intelligence in the Academic Literature: An Analysis of the Academ-AI Dataset

Towards the relationship between AIGC in manuscript writing and author profiles: evidence from preprints in LLMs

Quantitative Analysis of AI-Generated Texts in Academic Research: A Study of AI Presence in Arxiv Submissions using AI Detection Tool

A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT

What does Artificial Intelligence Generated Content bring to Teaching and Learning? A literature review on AIGC in Education

Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities

A Chinese perspective on artificial intelligence generated content and copyright

Unmasking artificial intelligence (AI): Identifying articles written by AI models

AI-Generated Content (AIGC): A Survey

Editorial: Generative artificial intelligence as a plagiarism problem

Artificial intelligence in applied (linguistics): a content analysis and future prospects

An investigation into the scientific landscape of the conversational and generative artificial intelligence, and human-chatbot interaction in education and research

Is AI Changing the Rules of Academic Misconduct? An In-depth Look at Students' Perceptions of 'AI-giarism'

Artificial intelligence (AI) content detection in ASCO scientific abstracts from 2021 to 2023.

A Survey on ChatGPT: AI-Generated Contents, Challenges, and Solutions

Examining the Prevalence and Dynamics of AI-Generated Media in Art Subreddits

The Evolution and Future Perspectives of Artificial Intelligence Generated Content

The Impact of AI on Academic Research and Publishing

Staying vigilant in the Age of AI: From content generation to content authentication