تقرير
CLEME2.0: Towards More Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction
| العنوان: | CLEME2.0: Towards More Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction |
|---|---|
| المؤلفون: | Ye, Jingheng, Xu, Zishan, Li, Yinghui, Cheng, Xuxin, Song, Linlin, Zhou, Qingyu, Zheng, Hai-Tao, Shen, Ying, Su, Xin |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computation and Language |
| الوصف: | The paper focuses on improving the interpretability of Grammatical Error Correction (GEC) metrics, which receives little attention in previous studies. To bridge the gap, we propose CLEME2.0, a reference-based evaluation strategy that can describe four elementary dimensions of GEC systems, namely hit-correction, error-correction, under-correction, and over-correction. They collectively contribute to revealing the critical characteristics and locating drawbacks of GEC systems. Evaluating systems by Combining these dimensions leads to high human consistency over other reference-based and reference-less metrics. Extensive experiments on 2 human judgement datasets and 6 reference datasets demonstrate the effectiveness and robustness of our method. All the codes will be released after the peer review. Comment: 16 pages, 8 tables, 2 figures. Under review |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2407.00934 |
| رقم الأكسشن: | edsarx.2407.00934 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |