دورية أكاديمية
ChatGPT Versus Consultants: Blinded Evaluation on Answering Otorhinolaryngology Case–Based Questions
العنوان: | ChatGPT Versus Consultants: Blinded Evaluation on Answering Otorhinolaryngology Case–Based Questions |
---|---|
المؤلفون: | Christoph Raphael Buhr, Harry Smith, Tilman Huppertz, Katharina Bahr-Hamm, Christoph Matthias, Andrew Blaikie, Tom Kelsey, Sebastian Kuhn, Jonas Eckrich |
المصدر: | JMIR Medical Education, Vol 9, p e49183 (2023) |
بيانات النشر: | JMIR Publications, 2023. |
سنة النشر: | 2023 |
المجموعة: | LCC:Special aspects of education LCC:Medicine (General) |
مصطلحات موضوعية: | Special aspects of education, LC8-6691, Medicine (General), R5-920 |
الوصف: | BackgroundLarge language models (LLMs), such as ChatGPT (Open AI), are increasingly used in medicine and supplement standard search engines as information sources. This leads to more “consultations” of LLMs about personal medical symptoms. ObjectiveThis study aims to evaluate ChatGPT’s performance in answering clinical case–based questions in otorhinolaryngology (ORL) in comparison to ORL consultants’ answers. MethodsWe used 41 case-based questions from established ORL study books and past German state examinations for doctors. The questions were answered by both ORL consultants and ChatGPT 3. ORL consultants rated all responses, except their own, on medical adequacy, conciseness, coherence, and comprehensibility using a 6-point Likert scale. They also identified (in a blinded setting) if the answer was created by an ORL consultant or ChatGPT. Additionally, the character count was compared. Due to the rapidly evolving pace of technology, a comparison between responses generated by ChatGPT 3 and ChatGPT 4 was included to give an insight into the evolving potential of LLMs. ResultsRatings in all categories were significantly higher for ORL consultants (P |
نوع الوثيقة: | article |
وصف الملف: | electronic resource |
اللغة: | English |
تدمد: | 2369-3762 |
Relation: | https://mededu.jmir.org/2023/1/e49183; https://doaj.org/toc/2369-3762 |
DOI: | 10.2196/49183 |
URL الوصول: | https://doaj.org/article/b213d95a88da4b729d5d31d72068da98 |
رقم الأكسشن: | edsdoj.b213d95a88da4b729d5d31d72068da98 |
قاعدة البيانات: | Directory of Open Access Journals |
تدمد: | 23693762 |
---|---|
DOI: | 10.2196/49183 |