تقرير
Towards Red Teaming in Multimodal and Multilingual Translation
| العنوان: | Towards Red Teaming in Multimodal and Multilingual Translation |
|---|---|
| المؤلفون: | Ropers, Christophe, Dale, David, Hansanti, Prangthip, Gonzalez, Gabriel Mejia, Evtimov, Ivan, Wong, Corinne, Touret, Christophe, Pereyra, Kristina, Kim, Seohyun Sonia, Ferrer, Cristian Canton, Andrews, Pierre, Costa-jussà, Marta R. |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computation and Language, Computer Science - Computers and Society, I.2.7 |
| الوصف: | Assessing performance in Natural Language Processing is becoming increasingly complex. One particular challenge is the potential for evaluation datasets to overlap with training data, either directly or indirectly, which can lead to skewed results and overestimation of model performance. As a consequence, human evaluation is gaining increasing interest as a means to assess the performance and reliability of models. One such method is the red teaming approach, which aims to generate edge cases where a model will produce critical errors. While this methodology is becoming standard practice for generative AI, its application to the realm of conditional AI remains largely unexplored. This paper presents the first study on human-based red teaming for Machine Translation (MT), marking a significant step towards understanding and improving the performance of translation models. We delve into both human-based red teaming and a study on automation, reporting lessons learned and providing recommendations for both translation models and red teaming drills. This pioneering work opens up new avenues for research and development in the field of MT. Comment: arXiv admin note: substantial text overlap with arXiv:2312.05187 |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2401.16247 |
| رقم الأكسشن: | edsarx.2401.16247 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |