Use of a Large Language Model as a Dermatology Case Narrator: Exploring the Dynamics of a Chatbot as an Educational Tool in Dermatology

Introduction

Artificial intelligence (AI) is no longer a futuristic concept but a present reality that has rapidly changed all aspects of life; health care is no exception. In medical education, AI offers tools that have the potential to outperform traditional methods of teaching and learning. Dermatology, a medical specialty that relies almost exclusively on visual recognition and clinical pattern analysis, provides fertile ground for AI to revolutionize how medical students and aspiring dermatologists are trained [1]. Chatbots, such as ChatGPT, can play a transformative role in the medical education of dermatology. They can serve as on-demand tutors, providing instant explanations for complex dermatological terms, clarifying concepts, or answering questions in real time. This capability allows medical students to explore topics and receive personalized support and guidance while studying. Along with supporting individual learning, the tools are also invaluable for educators, as they enable efficient creation of teaching materials. AI-driven models can generate quizzes, flashcards, summary notes, and even realistic and diverse clinical case scenarios [2,3]. The primary objective of this study was to assess whether large language models like ChatGPT-4 can create engaging, educational, and clinically relevant case-based scenarios for medical students.

Methods

Study Overview

Sixty-four medical students from two university-affiliated hospitals, Attikon University Hospital and University Hospital of Larissa, participated.

We assessed whether large language models like ChatGPT-4 can create engaging, educational, and clinically relevant case-based scenarios for medical students. “Engaging” refers to how much a case captures the learner’s interest, encourages active thinking, and motivates further exploration of the topic. Each AI-generated case was matched to a non-AI case on the same dermatological condition and similar educational objectives. Care was taken to ensure that both versions addressed comparable levels of difficulty, albeit with their inherent differences in style and formulation.

We developed a questionnaire featuring a mix of AI-generated and non-AI cases. The medical students were presented with multiple-choice questions, true/false statements, and correlation exercises. The AI-generated cases were created by instructing ChatGPT-4 to “generate an educational case in the form of multiple-choice questions for medical students” on the first attempt. Students graded each question on a Likert scale from 1 to 5, assessing how educational, motivational, or challenging they found the cases, without knowing the creator.

The final scores were evaluated regarding the normality with the Shapiro Wilk test, and then, the appropriate statistical tests were performed; for example, the Wilcoxon W test was the non-parametric test for paired data, as most comparisons were based on data with a significant departure from normality.

Ethical Considerations

Participation was entirely voluntary and anonymous. All the participants provided consent, and the included data were de-identified. Institutional Review Board approval was not applicable, as it relied exclusively on voluntary and anonymous questionnaire responses, without the collection of identifiable or sensitive personal information.

Results

Of 64 students, 45 answered the questionnaire (response rate of 70.3%). Among the 45 students, 36 (80%) reported using ChatGPT-4, 27 (60%) mentioned using Gemini-AI, while 9 (20%) students indicated they had never used a chatbot (the students could provide more than one answer). Twenty-five students (56%) stated that they used chatbots in their studies, though none reported using them in clinical practice.

Non-AI cases are thought to be more educational, motivational, and challenging in most scenarios, with statistical significance in many cases (Table 1). In contrast, in the case of True or False statements exercise and correlation test, no differences were detected between AI and non-AI examples. In the questionnaire students were asked about a conflicting situation involving multiple learning resources, including AI-based tools and traditional sources that provide different answers to the same question, 6.7% of students (3/45) would trust the AI answer, 68.9% of students (31/45) would trust the Internet source, and 24.4% of students (11/45) would further discuss the topic with a tutor.

Discussion

In dermatology, no studies have directly compared AI-generated case scenarios with those authored by clinicians in terms of the educational value, clinical relevance, or student engagement. In our study, AI-created cases were characterized by detailed case descriptions, analysis of medical history, clinical examinations, and follow-up questions but lacked the depth, clinical relevance, and motivational elements found in non-AI cases. Non-AI cases were shorter, presented clinical dilemmas, offered direct questions, and included challenging scenarios that students found more educational and engaging [4,5]. Non-AI questions, such as those comparing sebaceous hyperkeratosis and melanoma, differentiating between atopic dermatitis and scabies in a child patient, searching for comorbidities in rosacea, and critically evaluating a patient with hidradenitis suppurativa, were assessed with higher scores compared to the straightforward cases by AI chatbots. However, this observation does not imply that AI-generated questions have no value, as they can help students test their understanding. Tutors are also encouraged to make their own questions that simulate real-life scenarios that challenges students and do not rely blindly on chatbots for the quick production of exercises.