Abstract:Background: Text-generating artificial intelligence (AI) such as ChatGPT offers many opportunities and challenges in medical education. Acquiring practical skills necessary for using AI in a clinical context is crucial, especially for medical education. Objective: This explorative study aimed to investigate the feasibility of integrating ChatGPT into teaching units and to evaluate the course and the importance of AI-related competencies for medical students. Since a possible application of ChatGPT in the medical field could be the generation of information for patients, we further investigated how such information is perceived by students in terms of persuasiveness and quality. Methods: ChatGPT was integrated into 3 different teaching units of a blended learning course for medical students. Using a mixed methods approach, quantitative and qualitative data were collected. As baseline data, we assessed students' characteristics, including their openness to digital innovation. The students evaluated the integration of ChatGPT into the course and shared their thoughts regarding the future of text-generating AI in medical education. The course was evaluated based on the Kirkpatrick Model, with satisfaction, learning progress, and applicable knowledge considered as key assessment levels. In ChatGPT-integrating teaching units, students evaluated videos featuring information for patients regarding their persuasiveness on treatment expectations in a self-experience experiment and critically reviewed information for patients written using ChatGPT 3.5 based on different prompts. Results: A total of 52 medical students participated in the study. The comprehensive evaluation of the course revealed elevated levels of satisfaction, learning progress, and applicability specifically in relation to the ChatGPT-integrating teaching units. Furthermore, all evaluation levels demonstrated an association with each other. Higher openness to digital innovation was associated with higher satisfaction and, to a lesser extent, with higher applicability. AI-related competencies in other courses of the medical curriculum were perceived as highly important by medical students. Qualitative analysis highlighted potential use cases of ChatGPT in teaching and learning. In ChatGPT-integrating teaching units, students rated information for patients generated using a basic ChatGPT prompt as "moderate" in terms of comprehensibility, patient safety, and the correct application of communication rules taught during the course. The students' ratings were considerably improved using an extended prompt. The same text, however, showed the smallest increase in treatment expectations when compared with information provided by humans (patient, clinician, and expert) via videos. Conclusions: This study offers valuable insights into integrating the development of AI competencies into a blended learning course. Integration of ChatGPT enhanced learning experiences for medical students.

Exploring the Hurdles in Integrating Artificial Intelligence in Medical Education: A Talk with ChatGPT

The Rise of ChatGPT: Exploring its Potential in Medical Education

The Intersection of ChatGPT, Clinical Medicine, and Medical Education

ChatGPT and Generative Artificial Intelligence for Medical Education: Potential Impact and Opportunity

Embracing ChatGPT for Medical Education: Exploring Its Impact on Doctors and Medical Students

Challenge, integration, and change: ChatGPT and future anatomical education

Integrating AI in College Education: Positive yet Mixed Experiences with ChatGPT

[Reflections on the Implications of the Developments in ChatGPT for Changes in Medical Education Models]

Current Status of ChatGPT Use in Medical Education: Potentials, Challenges, and Strategies

ChatGPT in Medical Education and Research: A Boon or a Bane?

ChatGPT-4: Transforming Medical Education and Addressing Clinical Exposure Challenges in the Post-pandemic Era

Integration of ChatGPT Into a Course for Medical Students: Explorative Study on Teaching Scenarios, Students' Perception, and Applications

ChatGPT and artificial intelligence the future of healthcare?

Transformative effects of ChatGPT on modern education: Emerging Era of AI Chatbots

The tools of the future are the challenges of today: The use of ChatGPT in problem-based learning medical education

ChatGPT-A double-edged sword for healthcare education? Implications for assessments of dental students

ChatGPT in medicine: prospects and challenges: a review article

Enhancing Education with ChatGPT: Revolutionizing Personalized Learning and Teacher Support

ChatGPT for teachers: Practical examples for utilizing artificial intelligence for educational purposes

Artificial Intelligence and Objective Structured Clinical Examinations: Using ChatGPT to Revolutionize Clinical Skills Assessment in Medical Education