مؤتمر
Generating Multiple Choice Questions with a Multi-Angle Question Answering Model
| العنوان: | Generating Multiple Choice Questions with a Multi-Angle Question Answering Model |
|---|---|
| اللغة: | English |
| المؤلفون: | Olney, Andrew M. (ORCID |
| المصدر: | Grantee Submission. 2022. |
| Peer Reviewed: | Y |
| Page Count: | 7 |
| تاريخ النشر: | 2022 |
| Sponsoring Agency: | Institute of Education Sciences (ED) National Science Foundation (NSF) |
| Contract Number: | R305A190448 1918751 1934745 |
| نوع الوثيقة: | Speeches/Meeting Papers Reports - Research |
| Descriptors: | Test Construction, Multiple Choice Tests, Test Items, Models, Textbooks, Natural Language Processing |
| DOI: | 10.5281/zenodo.7761561 |
| مستخلص: | Multi-angle question answering models have recently been proposed that promise to perform related tasks like question generation. However, performance on related tasks has not been thoroughly studied. We investigate a leading model called Macaw on the task of multiple choice question generation and evaluate its performance on three angles that systematically reduce the complexity of the task. Our results indicate that despite the promise of generalization, Macaw performs poorly on untrained angles. Even on a trained angle, Macaw fails to generate four distinct multiple-choice options on 17% of inputs. We propose augmenting multiple-choice options by paraphrasing angle input and show this increases overall success to 97.5%. A human evaluation comparing the augmented multiple-choice questions with textbook questions on the same topic reveals that Macaw questions broadly score highly but below human questions. [This paper was published in: "Proceedings of the 3rd Workshop of the Learner Data Institute," edited by S. E. Fancsali and V. Rus, Learner Data Institute, 2022, pp. 18-23.] |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2023 |
| رقم الأكسشن: | ED630027 |
| قاعدة البيانات: | ERIC |
كن أول من يترك تعليقا!