دورية أكاديمية
A comparison of 3- and 4-option multiple-choice items for medical subspecialty in-training examinations.
| العنوان: | A comparison of 3- and 4-option multiple-choice items for medical subspecialty in-training examinations. |
|---|---|
| المؤلفون: | Chen D; The American Board of Anesthesiology, 4200 Six Forks Road, Suite 1100, Raleigh, NC, 27609, USA., Harman AE; The American Board of Anesthesiology, 4200 Six Forks Road, Suite 1100, Raleigh, NC, 27609, USA., Sun H; The American Board of Anesthesiology, 4200 Six Forks Road, Suite 1100, Raleigh, NC, 27609, USA. Huaping.Sun@theABA.org., Ye T; The American Board of Anesthesiology, 4200 Six Forks Road, Suite 1100, Raleigh, NC, 27609, USA., Gaiser RR; Yale School of Medicine, New Haven, CT, USA. |
| المصدر: | BMC medical education [BMC Med Educ] 2023 Apr 27; Vol. 23 (1), pp. 286. Date of Electronic Publication: 2023 Apr 27. |
| نوع المنشور: | Journal Article |
| اللغة: | English |
| بيانات الدورية: | Publisher: BioMed Central Country of Publication: England NLM ID: 101088679 Publication Model: Electronic Cited Medium: Internet ISSN: 1472-6920 (Electronic) Linking ISSN: 14726920 NLM ISO Abbreviation: BMC Med Educ Subsets: MEDLINE |
| أسماء مطبوعة: | Original Publication: London : BioMed Central, [2001- |
| مواضيع طبية MeSH: | Educational Measurement*/methods , Physical Examination*, Humans ; United States ; Child ; Reproducibility of Results |
| مستخلص: | Background: The American Board of Anesthesiology piloted 3-option multiple-choice items (MCIs) for its 2020 administration of 150-item subspecialty in-training examinations for Critical Care Medicine (ITE-CCM) and Pediatric Anesthesiology (ITE-PA). The 3-option MCIs were transformed from their 4-option counterparts, which were administered in 2019, by removing the least effective distractor. The purpose of this study was to compare physician performance, response time, and item and exam characteristics between the 4-option and 3-option exams. Methods: Independent-samples t-test was used to examine the differences in physician percent-correct score; paired t-test was used to examine the differences in response time and item characteristics. The Kuder and Richardson Formula 20 was used to calculate the reliability of each exam form. Both the traditional (distractor being selected by fewer than 5% of examinees and/or showing a positive correlation with total score) and sliding scale (adjusting the frequency threshold of distractor being chosen by item difficulty) methods were used to identify non-functioning distractors (NFDs). Results: Physicians who took the 3-option ITE-CCM (mean = 67.7%) scored 2.1 percent correct higher than those who took the 4-option ITE-CCM (65.7%). Accordingly, 3-option ITE-CCM items were significantly easier than their 4-option counterparts. No such differences were found between the 4-option and 3-option ITE-PAs (71.8% versus 71.7%). Item discrimination (4-option ITE-CCM [an average of 0.13], 3-option ITE-CCM [0.12]; 4-option ITE-PA [0.08], 3-option ITE-PA [0.09]) and exam reliability (0.75 and 0.74 for 4- and 3-option ITE-CCMs, respectively; 0.62 and 0.67 for 4-option and 3-option ITE-PAs, respectively) were similar between these two formats for both ITEs. On average, physicians spent 3.4 (55.5 versus 58.9) and 1.3 (46.2 versus 47.5) seconds less per item on 3-option items than 4-option items for ITE-CCM and ITE-PA, respectively. Using the traditional method, the percentage of NFDs dropped from 51.3% in the 4-option ITE-CCM to 37.0% in the 3-option ITE-CCM and from 62.7% to 46.0% for the ITE-PA; using the sliding scale method, the percentage of NFDs dropped from 36.0% to 21.7% for the ITE-CCM and from 44.9% to 27.7% for the ITE-PA. Conclusions: Three-option MCIs function as robustly as their 4-option counterparts. The efficiency achieved by spending less time on each item poses opportunities to increase content coverage for a fixed testing period. The results should be interpreted in the context of exam content and distribution of examinee abilities. (© 2023. The Author(s).) |
| References: | Acad Med. 2005 Oct;80(10 Suppl):S93-6. (PMID: 16199468) BMC Med Educ. 2010 Nov 24;10:85. (PMID: 21106066) Med Educ. 2014 Oct;48(10):1020-7. (PMID: 25200022) Nurse Educ Today. 2010 Aug;30(6):539-43. (PMID: 20053488) Natl Med J India. 2008 May-Jun;21(3):130-3. (PMID: 19004145) Adv Health Sci Educ Theory Pract. 2016 Aug;21(3):571-85. (PMID: 26597452) Adv Health Sci Educ Theory Pract. 2019 Mar;24(1):141-150. (PMID: 30362027) |
| فهرسة مساهمة: | Keywords: 3-option and 4-option multiple-choice items; Medical subspecialty in-training examinations; Non-functioning distractors; Psychometric properties of items and exams |
| تواريخ الأحداث: | Date Created: 20230427 Date Completed: 20230501 Latest Revision: 20230501 |
| رمز التحديث: | 20240628 |
| مُعرف محوري في PubMed: | PMC10134669 |
| DOI: | 10.1186/s12909-023-04277-2 |
| PMID: | 37106417 |
| قاعدة البيانات: | MEDLINE |
Find this article in full text from Springer Verlag. 
Full Text Finder | تدمد: | 1472-6920 |
|---|---|
| DOI: | 10.1186/s12909-023-04277-2 |