Preparation of iodine labeled recombinant human erythropoietin.

H. Kinoshita,N. Ohishi,A. Okazaki

1991-05-01

Abstract:125I-labelled recombinant human erythropoietin (125I-rh-EPO; CAS for EPO: 11096-26-7) was prepared by using iodo-gen (1,3,4,6-tetrachloro-3,6-diphenylglycouril), which showed a similar immunoreactivity to anti rh-EPO anti-rabbit serum and a similar pharmacokinetics with that of non-labeled rh-EPO.

What problem does this paper attempt to address?

Human versus machine: identifying ChatGPT-generated abstracts in Gynecology and Urogynecology

Evelyn T. Pan,Maria Florian-Rodriguez

DOI: https://doi.org/10.1016/j.ajog.2024.04.045

IF: 9.8

2024-05-07

American Journal of Obstetrics and Gynecology

Abstract:Background ChatGPT, a publicly available artificial intelligence (AI) large language model, has allowed for sophisticated AI technology on demand. Indeed, use of ChatGPT has already begun to make its way into medical research. However, the medical community has yet to understand the capabilities and ethical considerations of AI within this context, and unknowns exist regarding ChatGPT's writing abilities, accuracy, and implications for authorship. Objectives We hypothesize that human reviewers and AI detection software differ in their ability to correctly identify original published abstracts and AI-written abstracts in the subjects of Gynecology and Urogynecology. We additionally suspect that concrete differences in writing errors, readability, and perceived writing quality exist between original and AI-generated text. Study Design Twenty-five articles published in high impact medical journals and a collection of Gynecology and Urogynecology journals were selected. ChatGPT was prompted to write 25 corresponding AI-generated abstracts, providing the abstract title, journal-dictated abstract requirements, and select original results. The original and AI-generated abstracts were reviewed by blinded Gynecology and Urogynecology faculty and fellows to identify the writing as original or AI-generated. All abstracts were analyzed by publicly available AI detection software GPTZero, Originality, and Copyleaks and were assessed for writing errors and quality by AI writing assistant Grammarly. Results One hundred fifty-seven reviews of 25 original and 25 AI-generated abstracts were conducted by 26 faculty and 4 fellows. Fifty-seven percent of original abstracts and 42.3% of AI-generated abstracts were correctly identified for an average of 49.7% across all abstracts. All three AI detectors rated the original abstracts as less likely be AI-written than the ChatGPT-generated abstracts (GPTZero 5.8 vs 73.3%, p<0.001; Originality 10.9 vs 98.1%, p<0.001; Copyleaks 18.6 vs 58.2%, p<0.001). The performance of the three AI detection software differed when analyzing all abstracts (p=0.03), original abstracts (p<0.001), and AI-generated abstracts (p<0.001). Grammarly text analysis identified more writing issues and correctness errors in original than AI abstracts, including lower Grammarly score reflective of poorer writing quality (82.3 vs 88.1, p=0.006), more total writing issues (19.2 vs 12.8, p<0.001), critical issues (5.4 vs 1.3, p<0.001), confusing words (0.8 vs 0.1, p=0.006), misspelled words (1.7 vs 0.6, p=0.02), incorrect determiner use (1.2 vs 0.2, p=0.002), and comma misuse (0.3 vs 0.0, p=0.005). Conclusions Human reviewers are unable to detect the subtle differences between human and ChatGPT-generated scientific writing due to AI's ability to generate tremendously realistic text. AI detection software improve identification of AI-generated writing but still lack complete accuracy and require programmatic improvements in order to achieve optimal detection. As reviewers and editors may be unable to reliably detect AI-generated pieces, clear guidelines for reporting AI use by authors and implementing AI detection software in the review process will need to be established as AI chatbots gain more widespread use.

obstetrics & gynecology
Assessing the Reproducibility of the Structured Abstracts Generated by ChatGPT and Bard Compared to Human-Written Abstracts in the Field of Spine Surgery: Comparative Analysis

Hong Jin Kim,Jae Hyuk Yang,Dong-Gune Chang,Lawrence G Lenke,Javier Pizones,René Castelein,Kota Watanabe,Per D Trobisch,Gregory M Mundis Jr,Seung Woo Suh,Se-Il Suk

DOI: https://doi.org/10.2196/52001

IF: 7.076

2024-06-27

Journal of Medical Internet Research

Abstract:Background: Due to recent advances in artificial intelligence (AI), language model applications can generate logical text output that is difficult to distinguish from human writing. ChatGPT (OpenAI) and Bard (subsequently rebranded as "Gemini"; Google AI) were developed using distinct approaches, but little has been studied about the difference in their capability to generate the abstract. The use of AI to write scientific abstracts in the field of spine surgery is the center of much debate and controversy. Objective: The objective of this study is to assess the reproducibility of the structured abstracts generated by ChatGPT and Bard compared to human-written abstracts in the field of spine surgery. Methods: In total, 60 abstracts dealing with spine sections were randomly selected from 7 reputable journals and used as ChatGPT and Bard input statements to generate abstracts based on supplied paper titles. A total of 174 abstracts, divided into human-written abstracts, ChatGPT-generated abstracts, and Bard-generated abstracts, were evaluated for compliance with the structured format of journal guidelines and consistency of content. The likelihood of plagiarism and AI output was assessed using the iThenticate and ZeroGPT programs, respectively. A total of 8 reviewers in the spinal field evaluated 30 randomly extracted abstracts to determine whether they were produced by AI or human authors. Results: The proportion of abstracts that met journal formatting guidelines was greater among ChatGPT abstracts (34/60, 56.6%) compared with those generated by Bard (6/54, 11.1%; P <.001). However, a higher proportion of Bard abstracts (49/54, 90.7%) had word counts that met journal guidelines compared with ChatGPT abstracts (30/60, 50%; P <.001). The similarity index was significantly lower among ChatGPT-generated abstracts (20.7%) compared with Bard-generated abstracts (32.1%; P <.001). The AI-detection program predicted that 21.7% (13/60) of the human group, 63.3% (38/60) of the ChatGPT group, and 87% (47/54) of the Bard group were possibly generated by AI, with an area under the curve value of 0.863 ( P <.001). The mean detection rate by human reviewers was 53.8% (SD 11.2%), achieving a sensitivity of 56.3% and a specificity of 48.4%. A total of 56.3% (63/112) of the actual human-written abstracts and 55.9% (62/128) of AI-generated abstracts were recognized as human-written and AI-generated by human reviewers, respectively. Conclusions: Both ChatGPT and Bard can be used to help write abstracts, but most AI-generated abstracts are currently considered unethical due to high plagiarism and AI-detection rates. ChatGPT-generated abstracts appear to be superior to Bard-generated abstracts in meeting journal formatting guidelines. Because humans are unable to accurately distinguish abstracts written by humans from those produced by AI programs, it is crucial to exercise special caution and examine the ethical boundaries of using AI programs, including ChatGPT and Bard.

health care sciences & services,medical informatics
Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers

Alexander T. Pearson,Frederick M. Howard,N. Markov,Yuan Luo,C. Gao,E. Dyer,S. Ramesh

DOI: https://doi.org/10.1101/2022.12.23.521610

2022-12-27

bioRxiv

Abstract:Background Large language models such as ChatGPT can produce increasingly realistic text, with unknown information on the accuracy and integrity of using these models in scientific writing. Methods We gathered ten research abstracts from five high impact factor medical journals (n=50) and asked ChatGPT to generate research abstracts based on their titles and journals. We evaluated the abstracts using an artificial intelligence (AI) output detector, plagiarism detector, and had blinded human reviewers try to distinguish whether abstracts were original or generated. Results All ChatGPT-generated abstracts were written clearly but only 8% correctly followed the specific journal’s formatting requirements. Most generated abstracts were detected using the AI output detector, with scores (higher meaning more likely to be generated) of median [interquartile range] of 99.98% [12.73, 99.98] compared with very low probability of AI-generated output in the original abstracts of 0.02% [0.02, 0.09]. The AUROC of the AI output detector was 0.94. Generated abstracts scored very high on originality using the plagiarism detector (100% [100, 100] originality). Generated abstracts had a similar patient cohort size as original abstracts, though the exact numbers were fabricated. When given a mixture of original and general abstracts, blinded human reviewers correctly identified 68% of generated abstracts as being generated by ChatGPT, but incorrectly identified 14% of original abstracts as being generated. Reviewers indicated that it was surprisingly difficult to differentiate between the two, but that the generated abstracts were vaguer and had a formulaic feel to the writing. Conclusion ChatGPT writes believable scientific abstracts, though with completely generated data. These are original without any plagiarism detected but are often identifiable using an AI output detector and skeptical human reviewers. Abstract evaluation for journals and medical conferences must adapt policy and practice to maintain rigorous scientific standards; we suggest inclusion of AI output detectors in the editorial process and clear disclosure if these technologies are used. The boundaries of ethical and acceptable use of large language models to help scientific writing remain to be determined.

Medicine,Biology,Computer Science
Association of reviewer experience with discriminating human-written versus ChatGPT-written abstracts

Gabriel Levin,Rene Pareja,David Viveros-Carreño,Emmanuel Sanchez Diaz,Elise Mann Yates,Behrouz Zand,Pedro T Ramirez

DOI: https://doi.org/10.1136/ijgc-2023-005162

2024-05-07

International Journal of Gynecological Cancer

Abstract:Objective To determine if reviewer experience impacts the ability to discriminate between human-written and ChatGPT-written abstracts. Methods Thirty reviewers (10 seniors, 10 juniors, and 10 residents) were asked to differentiate between 10 ChatGPT-written and 10 human-written (fabricated) abstracts. For the study, 10 gynecologic oncology abstracts were fabricated by the authors. For each human-written abstract we generated a ChatGPT matching abstract by using the same title and the fabricated results of each of the human generated abstracts. A web-based questionnaire was used to gather demographic data and to record the reviewers' evaluation of the 20 abstracts. Comparative statistics and multivariable regression were used to identify factors associated with a higher correct identification rate. Results The 30 reviewers discriminated 20 abstracts, giving a total of 600 abstract evaluations. The reviewers were able to correctly identify 300/600 (50%) of the abstracts: 139/300 (46.3%) of the ChatGPT-generated abstracts and 161/300 (53.7%) of the human-written abstracts (p=0.07). Human-written abstracts had a higher rate of correct identification (median (IQR) 56.7% (49.2–64.1%) vs 45.0% (43.2–48.3%), p=0.023). Senior reviewers had a higher correct identification rate (60%) than junior reviewers and residents (45% each; p=0.043 and p=0.002, respectively). In a linear regression model including the experience level of the reviewers, familiarity with artificial intelligence (AI) and the country in which the majority of medical training was achieved (English speaking vs non-English speaking), the experience of the reviewer (β=10.2 (95% CI 1.8 to 18.7)) and familiarity with AI (β=7.78 (95% CI 0.6 to 15.0)) were independently associated with the correct identification rate (p=0.019 and p=0.035, respectively). In a correlation analysis the number of publications by the reviewer was positively correlated with the correct identification rate (r28)=0.61, p<0.001. Conclusion A total of 46.3% of abstracts written by ChatGPT were detected by reviewers. The correct identification rate increased with reviewer and publication experience.

oncology,obstetrics & gynecology
What is the rate of text generated by artificial intelligence over a year of publication in Orthopedics & Traumatology: Surgery & Research? Analysis of 425 articles before versus after the launch of ChatGPT in November 2022

Théophile Bisi,Anthony Risser,Philippe Clavert,Henri Migaud,Julien Dartus

DOI: https://doi.org/10.1016/j.otsr.2023.103694

Abstract:Background: The use of artificial intelligence (AI) is soaring, and the launch of ChatGPT in November 2022 has accelerated this trend. This "chatbot" can generate complete scientific articles, with risk of plagiarism by mining existing data or downright fraud by fabricating studies with no real data at all. There are tools that detect AI in publications, but to our knowledge they have not been systematically assessed for publication in scientific journals. We therefore conducted a retrospective study on articles published in Orthopaedics & Traumatology: Surgery & Research (OTSR): firstly, to screen for AI-generated content before and after the publicized launch of ChatGPT; secondly, to assess whether AI was more often used in some countries than others to generate content; thirdly, to determine whether plagiarism rate correlated with AI-generation, and lastly, to determine whether elements other than text generation, and notably the translation procedure, could raise suspicion of AI use. Hypothesis: The rate of AI use increased after the publicized launch of ChatGPT v3.5 in November 2022. Material and methods: In all, 425 articles published between February 2022 and September 2023 (221 before and 204 after November 1, 2022) underwent ZeroGPT assessment of the level of AI generation in the final English-language version (abstract and body of the article). Two scores were obtained: probability of AI generation, in six grades from Human to AI; and percentage AI generation. Plagiarism was assessed on the Ithenticate application at submission. Articles in French were assessed in their English-language version as translated by a human translator, with comparison to automatic translation by Google Translate and DeepL. Results: AI-generated text was detected mainly in Abstracts, with a 10.1% rate of AI or considerable AI generation, compared to only 1.9% for the body of the article and 5.6% for the total body+abstract. Analysis for before and after November 2022 found an increase in AI generation in body+abstract, from 10.30±15.95% (range, 0-100%) to 15.64±19.8% (range, 0-99.93) (p < 0.04; NS for abstracts alone). AI scores differed between types of article: 14.9% for original articles and 9.8% for reviews (p<0.01). The highest rates of probable AI generation were in articles from Japan, China, South America and English-speaking countries (p<0.0001). Plagiarism rates did not increase between the two study periods, and were unrelated to AI rates. On the other hand, when articles were classified as "suspected" of AI generation (plagiarism rate ≥ 20%) or "non-suspected" (rate<20%), the "similarity" score was higher in suspect articles: 25.7±13.23% (range, 10-69%) versus 16.28±10% (range, 0-79%) (p < 0.001). In the body of the article, use of translation software was associated with higher AI rates than with a human translator: 3.5±5% for human translators, versus 18±10% and 21.9±11% respectively for Google Translate and DeepL (p < 0.001). Discussion: The present study revealed an increasing rate of AI use in articles published in OTSR. AI grades differed according to type of article and country of origin. Use of translation software increased the AI grade. In the long run, use of ChatGPT incurs a risk of plagiarism and scientific misconduct, and needs to be detected and signaled by a digital tag on any robot-generated text. Level of evidence: III; case-control study.
Editorial Commentary: Experts in Shoulder Surgery Do Not Consistently Detect Artificial Intelligence-Generated Scientific Abstracts

Mikalyn T DeFoor,Andrew J Sheean

DOI: https://doi.org/10.1016/j.arthro.2024.08.038

2024-09-05

Abstract:There has been an exponential growth in the number of artificial intelligence (AI) and machine learning (ML)-related publications in recent years. For example, in shoulder and elbow surgery, there was a 6-fold increase in the number of publications between 2018 and 2021. AI shows potential to improve diagnostic precision, generate precise surgical templates, direct personalized treatment plans, and reduce administrative costs. However, while AI and ML technologies have the ability to positively impact biomedical research, it should be closely monitored and used with extreme caution in the realm of research and scientific writing. Of concern, current large language models raise concerns regarding veracity of AI-generated content, copyright and ownership infringement, fabricated references, lack of in-text citation, plagiarism, and questions of authorship. Recent research shows that even the most experienced surgeons are unable to consistently detect AI-generated scientific writing. Of note, AI detection software is more adept in this role. AI should be used with caution in the development and production of scholarly work.
The ChatGPT conundrum: Human-generated scientific manuscripts misidentified as AI creations by AI text detection tool

Hooman H. Rashidi,Brandon D. Fennell,Samer Albahra,Bo Hu,Tom Gorbett

DOI: https://doi.org/10.1016/j.jpi.2023.100342

2023-01-01

Journal of Pathology Informatics

Abstract:AI Chat Bots such as ChatGPT are revolutionizing our AI capabilities, especially in text generation, to help expedite many tasks, but they introduce new dilemmas. The detection of AI-generated text has become a subject of great debate considering the AI text detector’s known and unexpected limitations. Thus far, much research in this area has focused on the detection of AI-generated text; however, the goal of this study was to evaluate the opposite scenario, an AI-text detection tool's ability to discriminate human-generated text. Thousands of abstracts from several of the most well-known scientific journals were used to test the predictive capabilities of these detection tools, assessing abstracts from 1980 to 2023. We found that the AI text detector erroneously identified up to 8% of the known real abstracts as AI-generated text. This further highlights the current limitations of such detection tools and argues for novel detectors or combined approaches that can address this shortcoming and minimize its unanticipated consequences as we navigate this new AI landscape.
Comparisons of Quality, Correctness, and Similarity Between ChatGPT-Generated and Human-Written Abstracts for Basic Research: Cross-Sectional Study

Shu-Li Cheng,Shih-Jen Tsai,Ya-Mei Bai,Chih-Hung Ko,Chih-Wei Hsu,Fu-Chi Yang,Chia-Kuang Tsai,Yu-Kang Tu,Szu-Nian Yang,Ping-Tao Tseng,Tien-Wei Hsu,Chih-Sung Liang,Kuan-Pin Su

DOI: https://doi.org/10.2196/51229

2023-12-25

Abstract:Background: ChatGPT may act as a research assistant to help organize the direction of thinking and summarize research findings. However, few studies have examined the quality, similarity (abstracts being similar to the original one), and accuracy of the abstracts generated by ChatGPT when researchers provide full-text basic research papers. Objective: We aimed to assess the applicability of an artificial intelligence (AI) model in generating abstracts for basic preclinical research. Methods: We selected 30 basic research papers from Nature, Genome Biology, and Biological Psychiatry. Excluding abstracts, we inputted the full text into ChatPDF, an application of a language model based on ChatGPT, and we prompted it to generate abstracts with the same style as used in the original papers. A total of 8 experts were invited to evaluate the quality of these abstracts (based on a Likert scale of 0-10) and identify which abstracts were generated by ChatPDF, using a blind approach. These abstracts were also evaluated for their similarity to the original abstracts and the accuracy of the AI content. Results: The quality of ChatGPT-generated abstracts was lower than that of the actual abstracts (10-point Likert scale: mean 4.72, SD 2.09 vs mean 8.09, SD 1.03; P<.001). The difference in quality was significant in the unstructured format (mean difference -4.33; 95% CI -4.79 to -3.86; P<.001) but minimal in the 4-subheading structured format (mean difference -2.33; 95% CI -2.79 to -1.86). Among the 30 ChatGPT-generated abstracts, 3 showed wrong conclusions, and 10 were identified as AI content. The mean percentage of similarity between the original and the generated abstracts was not high (2.10%-4.40%). The blinded reviewers achieved a 93% (224/240) accuracy rate in guessing which abstracts were written using ChatGPT. Conclusions: Using ChatGPT to generate a scientific abstract may not lead to issues of similarity when using real full texts written by humans. However, the quality of the ChatGPT-generated abstracts was suboptimal, and their accuracy was not 100%.
Synaptic potentials of digastric-muscle motoneurons

E. V. Gura,Y. P. Limanskii,A. I. Pilyavskii

DOI: https://doi.org/10.1007/BF01065589

Abstract:
Human versus artificial intelligence‐generated arthroplasty literature: A single‐blinded analysis of perceived communication, quality, and authorship source

Kyle W. Lawrence,Akram A. Habibi,Spencer A. Ward,Claudette M. Lajam,Ran Schwarzkopf,Joshua C. Rozell

DOI: https://doi.org/10.1002/rcs.2621

2024-02-14

International Journal of Medical Robotics and Computer Assisted Surgery

Abstract:Abstract Background Large language models (LLM) have unknown implications for medical research. This study assessed whether LLM‐generated abstracts are distinguishable from human‐written abstracts and to compare their perceived quality. Methods The LLM ChatGPT was used to generate 20 arthroplasty abstracts (AI‐generated) based on full‐text manuscripts, which were compared to originally published abstracts (human‐written). Six blinded orthopaedic surgeons rated abstracts on overall quality, communication, and confidence in the authorship source. Authorship‐confidence scores were compared to a test value representing complete inability to discern authorship. Results Modestly increased confidence in human authorship was observed for human‐written abstracts compared with AI‐generated abstracts ( p = 0.028), though AI‐generated abstract authorship‐confidence scores were statistically consistent with inability to discern authorship ( p = 0.999). Overall abstract quality was higher for human‐written abstracts ( p = 0.019). Conclusions AI‐generated abstracts' absolute authorship‐confidence ratings demonstrated difficulty in discerning authorship but did not achieve the perceived quality of human‐written abstracts. Caution is warranted in implementing LLMs into scientific writing.

surgery
The great detectives: humans versus AI detectors in catching large language model-generated medical writing

Jae Q. J. Liu,Kelvin T. K. Hui,Fadi Al Zoubi,Zing Z. X. Zhou,Dino Samartzis,Curtis C. H. Yu,Jeremy R. Chang,Arnold Y. L. Wong

DOI: https://doi.org/10.1007/s40979-024-00155-6

2024-05-20

International Journal for Educational Integrity

Abstract:Abstract Background The application of artificial intelligence (AI) in academic writing has raised concerns regarding accuracy, ethics, and scientific rigour. Some AI content detectors may not accurately identify AI-generated texts, especially those that have undergone paraphrasing. Therefore, there is a pressing need for efficacious approaches or guidelines to govern AI usage in specific disciplines. Objective Our study aims to compare the accuracy of mainstream AI content detectors and human reviewers in detecting AI-generated rehabilitation-related articles with or without paraphrasing. Study design This cross-sectional study purposively chose 50 rehabilitation-related articles from four peer-reviewed journals, and then fabricated another 50 articles using ChatGPT. Specifically, ChatGPT was used to generate the introduction, discussion, and conclusion sections based on the original titles, methods, and results. Wordtune was then used to rephrase the ChatGPT-generated articles. Six common AI content detectors (Originality.ai, Turnitin, ZeroGPT, GPTZero, Content at Scale, and GPT-2 Output Detector) were employed to identify AI content for the original, ChatGPT-generated and AI-rephrased articles. Four human reviewers (two student reviewers and two professorial reviewers) were recruited to differentiate between the original articles and AI-rephrased articles, which were expected to be more difficult to detect. They were instructed to give reasons for their judgements. Results Originality.ai correctly detected 100% of ChatGPT-generated and AI-rephrased texts. ZeroGPT accurately detected 96% of ChatGPT-generated and 88% of AI-rephrased articles. The areas under the receiver operating characteristic curve (AUROC) of ZeroGPT were 0.98 for identifying human-written and AI articles. Turnitin showed a 0% misclassification rate for human-written articles, although it only identified 30% of AI-rephrased articles. Professorial reviewers accurately discriminated at least 96% of AI-rephrased articles, but they misclassified 12% of human-written articles as AI-generated. On average, students only identified 76% of AI-rephrased articles. Reviewers identified AI-rephrased articles based on ‘incoherent content’ (34.36%), followed by ‘grammatical errors’ (20.26%), and ‘insufficient evidence’ (16.15%). Conclusions and relevance This study directly compared the accuracy of advanced AI detectors and human reviewers in detecting AI-generated medical writing after paraphrasing. Our findings demonstrate that specific detectors and experienced reviewers can accurately identify articles generated by Large Language Models, even after paraphrasing. The rationale employed by our reviewers in their assessments can inform future evaluation strategies for monitoring AI usage in medical education or publications. AI content detectors may be incorporated as an additional screening tool in the peer-review process of academic journals.
Acquired subglottic stenosis following prolonged endotracheal intubation. A canine model.

J. Supance,J. Reilly,W. Doyle,C. Bluestone,J. Hubbard

DOI: https://doi.org/10.1001/ARCHOTOL.1982.00790590049014

1982-11-01

A M A Archives of Otolaryngology

Abstract:We developed a canine animal model of subglottic stenosis following prolonged intubation with modified non-cuffed endotracheal tubes. None of the puppies intubated for seven days had an irreversible stenosing subglottic lesion, whereas all animals intubated for 14 days or more had at least a 40% to 50% reduction of the subglottic lumen secondary to maturing fibrotic stenosis. The model described is more congruent with the known and suspected pathogenesis in those infants and children who require prolonged endotracheal intubation and subsequently acquire subglottic stenosis, and can be used in evaluating the efficacy of medical therapy or surgical therapy, or both, in the prevention or management of this disease.
Relationship Status as an Influence on Cybersex Activity: Cybersex, Youth, and Steady Partner

R. Ballester-Arnal,J. Castro-Calvo,M. Gil-Llario,C. Giménez-García

DOI: https://doi.org/10.1080/0092623X.2013.772549

2014-09-03

Journal of Sex & Marital Therapy

Abstract:The authors focus on the influence of participants’ having or not having a steady partner when reference to cybersex use. Participants were 1,239 young, Spanish individuals who completed the Internet Sex Screening Test. Results showed the influence of being in a relationship on certain consumption dimensions of cybersex; the influence was found to be greater in men than in women. In general, cybersex activity was higher for single participants, although it was also significant for participants with a steady partner. The authors’ findings facilitate the comprehension of the effect of new technologies in intimate human relationships.
Rise of the Machines: The Prevalence and Disclosure of Artificial Intelligence–Generated Text in High-Impact Orthopaedic Journals

Benjamin D. Pesante,Cyril Mauffrey,Joshua A. Parry

DOI: https://doi.org/10.5435/jaaos-d-24-00318

IF: 4

2024-06-27

Journal of the American Academy of Orthopaedic Surgeons

Abstract:Introduction: While most orthopaedic journals permit the use of artificial intelligence (AI) in article development, they require that AI not be listed as an author, that authors take full responsibility for its accuracy, and that AI use be disclosed. This study aimed to assess the prevalence and disclosure of AI-generated text in abstracts published in high-impact orthopaedic journals. Methods: Abstracts published from January 1, 2024, to February 19, 2024, in five orthopaedic journals were analyzed: the American Journal of Sports Medicine; the Journal of Arthroplasty; the Journal of Bone and Joint Surgery; the Knee Surgery, Sports, Traumatology, and Arthroscopy (KSSTA) journal; and the BMC Musculoskeletal Disorders (BMC MD) journal. Artificial intelligence detection software was used to evaluate each abstract for AI-generated text. Disclosure of AI use, country of origin, and article type (clinical, preclinical, review, or AI/machine learning) were documented. To evaluate the accuracy of AI detection software, 60 consecutive articles published in the Journal of Bone and Joint Surgery in 2014, before AI writing software was available, were also evaluated. These abstracts were evaluated again after being rewritten with AI writing software. The sensitivity and specificity of the software program for AI-generated text were calculated. Results: A total of 577 abstracts were included in the analysis. AI-generated text was detected in 4.8% of abstracts, ranging from 0% to 12% by journal. Only one (3.6%) of the 28 abstracts with AI-generated text disclosed its use. Abstracts with AI-generated text were more likely to be from the Asian continent (57.1% vs. 28.0%, P = 0.001) and to involve topics of AI or machine learning (21.4% vs. 0.6%, P < 0.0001). The sensitivity and specificity of the AI detection software program were determined to be 91.7% (55/60) and 100% (60/60). Discussion: A small percentage of abstracts published in high-impact orthopaedic journals contained AI-generated text, and most did not report the use of AI despite journal requirements. Level of evidence: Diagnostic Level III.

surgery,orthopedics
Digital Ink and Surgical Dreams: Perceptions of Artificial Intelligence-Generated Essays in Residency Applications

Loralai M Crawford,Peter Hendzlik,Justine Lam,Lisa M Cannon,Yanjie Qi,Lauren DeCaporale-Ryan,Nicole A Wilson

DOI: https://doi.org/10.1016/j.jss.2024.06.020

2024-07-22

Abstract:Introduction: Large language models like Chat Generative Pre-Trained Transformer (ChatGPT) are increasingly used in academic writing. Faculty may consider use of artificial intelligence (AI)-generated responses a form of cheating. We sought to determine whether general surgery residency faculty could detect AI versus human-written responses to a text prompt; hypothesizing that faculty would not be able to reliably differentiate AI versus human-written responses. Methods: Ten essays were generated using a text prompt, "Tell us in 1-2 paragraphs why you are considering the University of Rochester for General Surgery residency" (Current trainees: n = 5, ChatGPT: n = 5). Ten blinded faculty reviewers rated essays (ten-point Likert scale) on the following criteria: desire to interview, relevance to the general surgery residency, overall impression, and AI- or human-generated; with scores and identification error rates compared between the groups. Results: There were no differences between groups for %total points (ChatGPT 66.0 ± 13.5%, human 70.0 ± 23.0%, P = 0.508) or identification error rates (ChatGPT 40.0 ± 35.0%, human 20.0 ± 30.0%, P = 0.175). Except for one, all essays were identified incorrectly by at least two reviewers. Essays identified as human-generated received higher overall impression scores (area under the curve: 0.82 ± 0.04, P < 0.01). Conclusions: Whether use of AI tools for academic purposes should constitute academic dishonesty is controversial. We demonstrate that human and AI-generated essays are similar in quality, but there is bias against presumed AI-generated essays. Faculty are not able to reliably differentiate human from AI-generated essays, thus bias may be misdirected. AI-tools are becoming ubiquitous and their use is not easily detected. Faculty must expect these tools to play increasing roles in medical education.
Inclusion of Women and Minorities as Subjects in Clinical Research--Opportunities and Challenges.

B. Seto

Abstract:
Artificial intelligence (AI) content detection in ASCO scientific abstracts from 2021 to 2023.

Frederick Matthew Howard,Anran Li,Mark Riffon,Elizabeth Garrett-Mayer,Alexander T. Pearson

DOI: https://doi.org/10.1200/jco.2024.42.16_suppl.1565

IF: 45.3

2024-06-01

Journal of Clinical Oncology

Abstract:1565 Background: Generative AI models such as OpenAI’s ChatGPT have been broadly utilized throughout the medical literature. Previous studies have found that AI can generate scientific abstracts which can be difficult to distinguish from the work of human authors. There is a pressing need to characterize utilization of AI in scientific writing to guide policy. Methods: In collaboration with ASCO's Center for Research and Analytics, we extracted text from all scientific abstracts submitted to ASCO 2021 – 2023 Annual Meetings. Likelihood of AI content was evaluated by four AI detectors: GPTZero, Originality.ai (OGAI), Sapling, and Kashyap's AI Content Detector (AICD). Each detector produces a numeric likelihood of AI content. Predictions were dichotomized for uniform analysis across detectors: those in the top 5% for a given detector were considered as having high likelihood of AI content. Logistic regression was used to compute odds ratio (OR) for AI-generated content with respect to submission year and abstract characteristics. Predictions were also assessed for 10 human-written abstracts as negative controls, and 10 produced by OpenAI’s GPT-3 and GPT-4 models as positive controls. Results: 15,553 abstracts met inclusion criteria. 5,179 (33%) were published online only, and 5,327 (34%) referenced registered clinical trials. Across all detectors, abstracts submitted in 2023 were significantly more likely to contain AI content than those in 2021 (OR range 1.3 - 1.7). In abstracts from 2023, AI content score was associated with online only publication, lack of clinical trial number, and abstract track (Table). None of the 10 negative control human written abstracts were identified as AI generated, whereas 100%, 95%, 90%, and 30% of the GPT-3/4 generated abstracts were classified as AI generated by Sapling, GPTZero, OGAI, and AICD respectively using the 5% threshold. Additional results will be presented. Conclusions: AI content detectors uniformly suggest a higher likelihood of AI content generation for abstracts submitted in 2023. Predicted AI content is associated with triage of abstracts to online only presentation, suggesting predicted AI content is associated with lower perceived abstract quality. Further work is needed to understand the accuracy of AI detectors and utility in the abstract review process. [Table: see text]

oncology
Distinguishing GPT-4-generated Radiology Abstracts from Original Abstracts: Performance of Blinded Human Observers and AI Content Detector

Furkan Ufuk,Hakki Peker,Ergin Sagtas,Ahmet Baki Yagci

DOI: https://doi.org/10.1101/2023.04.28.23289283

2023-05-04

MedRxiv

Abstract:Objective: To determine GPT-4's effectiveness in writing scientific radiology article abstracts and investigate human reviewers' and AI Content detectors' success in distinguishing these abstracts. Additionally, to determine the similarity scores of abstracts generated by GPT-4 to better understand its ability to create unique text. Methods: The study collected 250 original articles published between 2021 and 2023 in five radiology journals. The articles were randomly selected, and their abstracts were generated by GPT-4 using a specific prompt. Three experienced academic radiologists independently evaluated the GPT-4 generated and original abstracts to distinguish them as original or generated by GPT-4. All abstracts were also uploaded to an AI Content Detector and plagiarism detector to calculate similarity scores. Statistical analysis was performed to determine discrimination performance and similarity scores. Results: Out of 134 GPT-4 generated abstracts, average of 75 (56%) were detected by reviewers, and average of 50 (43%) original abstracts were falsely categorized as GPT-4 generated abstracts by reviewers. The sensitivity, specificity, accuracy, PPV, and NPV of observers in distinguishing GPT-4 written abstracts ranged from 51.5% to 55.6%, 56.1% to 70%, 54.8% to 60.8%, 41.2% to 76.7%, and 47% to 62.7%, respectively. No significant difference was observed between observers in discrimination performance. Conclusion: GPT-4 can generate convincing scientific radiology article abstracts. However, human reviewers and AI Content detectors have difficulty in distinguishing GPT-4 generated abstracts from original ones.
Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts

Taesoon Hwang,Nishant Aggarwal,Pir Zarak Khan,Thomas Roberts,Amir Mahmood,Madlen M. Griffiths,Nick Parsons,Saboor Khan

DOI: https://doi.org/10.1371/journal.pone.0297701

IF: 3.7

2024-02-15

PLoS ONE

Abstract:ChatGPT, a sophisticated large language model (LLM), has garnered widespread attention for its ability to mimic human-like communication. As recent studies indicate a potential supportive role of ChatGPT in academic writing, we assessed the LLM's capacity to generate accurate and comprehensive scientific abstracts from published Randomised Controlled Trial (RCT) data, focusing on the adherence to the Consolidated Standards of Reporting Trials for s (CONSORT-A) statement, in comparison to the original authors' abstracts. RCTs, identified in a PubMed/MEDLINE search post-September 2021 across various medical disciplines, were subjected to abstract generation via ChatGPT versions 3.5 and 4, following the guidelines of the respective journals. The overall quality score (OQS) of each abstract was determined by the total number of adequately reported components from the 18-item CONSORT-A checklist. Additional outcome measures included percent adherence to each CONOSORT-A item, readability, hallucination rate, and regression analysis of reporting quality determinants. Original abstracts achieved a mean OQS of 11.89 (95% CI: 11.23–12.54), outperforming GPT 3.5 (7.89; 95% CI: 7.32–8.46) and GPT 4 (5.18; 95% CI: 4.64–5.71). Compared to GPT 3.5 and 4 outputs, original abstracts were more adherent with 10 and 14 CONSORT-A items, respectively. In blind assessments, GPT 3.5-generated abstracts were deemed most readable in 62.22% of cases which was significantly greater than the original (31.11%; P = 0.003) and GPT 4-generated (6.67%; P<0.001) abstracts. Moreover, ChatGPT 3.5 exhibited a hallucination rate of 0.03 items per abstract compared to 1.13 by GPT 4. No determinants for improved reporting quality were identified for GPT-generated abstracts. While ChatGPT could generate more readable abstracts, their overall quality was inferior to the original abstracts. Yet, its proficiency to concisely relay key information with minimal error holds promise for medical research and warrants further investigations to fully ascertain the LLM's applicability in this domain.

multidisciplinary sciences
Comparison of an AI-Generated Case Report With a Human-Written Case Report: Practical Considerations for AI-Assisted Medical Writing

Denver S Pinto,Sharon M Noronha,Gaurav Saigal,Robert M Quencer

DOI: https://doi.org/10.7759/cureus.60461

2024-05-16

Cureus

Abstract:Introduction: The utility of ChatGPT has recently caused consternation in the medical world. While it has been utilized to write manuscripts, only a few studies have evaluated the quality of manuscripts generated by AI (artificial intelligence). Objective: We evaluate the ability of ChatGPT to write a case report when provided with a framework. We also provide practical considerations for manuscript writing using AI. Methods: We compared a manuscript written by a blinded human author (10 years of medical experience) with a manuscript written by ChatGPT on a rare presentation of a common disease. We used multiple iterations of the manuscript generation request to derive the best ChatGPT output. Participants, outcomes, and measures: 22 human reviewers compared the manuscripts using parameters that characterize human writing and relevant standard manuscript assessment criteria, viz., scholarly impact quotient (SIQ). We also compared the manuscripts using the "average perplexity score" (APS), "burstiness score" (BS), and "highest perplexity of a sentence" (GPTZero parameters to detect AI-generated content). Results: The human manuscript had a significantly higher quality of presentation and nuanced writing (p<0.05). Both manuscripts had a logical flow. 12/22 reviewers were able to identify the AI-generated manuscript (p<0.05), but 4/22 reviewers wrongly identified the human-written manuscript as AI-generated. GPTZero software erroneously identified four sentences of the human-written manuscript to be AI-generated. Conclusion: Though AI showed an ability to highlight the novelty of the case report and project a logical flow comparable to the human manuscript, it could not outperform the human writer on all parameters. The human manuscript showed a better quality of presentation and more nuanced writing. The practical considerations we provide for AI-assisted medical writing will help to better utilize AI in manuscript writing.

Preparation of iodine labeled recombinant human erythropoietin.

Human versus machine: identifying ChatGPT-generated abstracts in Gynecology and Urogynecology

Assessing the Reproducibility of the Structured Abstracts Generated by ChatGPT and Bard Compared to Human-Written Abstracts in the Field of Spine Surgery: Comparative Analysis

Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers

Association of reviewer experience with discriminating human-written versus ChatGPT-written abstracts

What is the rate of text generated by artificial intelligence over a year of publication in Orthopedics & Traumatology: Surgery & Research? Analysis of 425 articles before versus after the launch of ChatGPT in November 2022

Editorial Commentary: Experts in Shoulder Surgery Do Not Consistently Detect Artificial Intelligence-Generated Scientific Abstracts

The ChatGPT conundrum: Human-generated scientific manuscripts misidentified as AI creations by AI text detection tool

Comparisons of Quality, Correctness, and Similarity Between ChatGPT-Generated and Human-Written Abstracts for Basic Research: Cross-Sectional Study

Synaptic potentials of digastric-muscle motoneurons

Human versus artificial intelligence‐generated arthroplasty literature: A single‐blinded analysis of perceived communication, quality, and authorship source

The great detectives: humans versus AI detectors in catching large language model-generated medical writing

Acquired subglottic stenosis following prolonged endotracheal intubation. A canine model.

Relationship Status as an Influence on Cybersex Activity: Cybersex, Youth, and Steady Partner

Rise of the Machines: The Prevalence and Disclosure of Artificial Intelligence–Generated Text in High-Impact Orthopaedic Journals

Digital Ink and Surgical Dreams: Perceptions of Artificial Intelligence-Generated Essays in Residency Applications

Inclusion of Women and Minorities as Subjects in Clinical Research--Opportunities and Challenges.

Artificial intelligence (AI) content detection in ASCO scientific abstracts from 2021 to 2023.

Distinguishing GPT-4-generated Radiology Abstracts from Original Abstracts: Performance of Blinded Human Observers and AI Content Detector

Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts

Comparison of an AI-Generated Case Report With a Human-Written Case Report: Practical Considerations for AI-Assisted Medical Writing