The Methods for Reducing the Number of OOVs in Chinese-Uyghur NMT System

التفاصيل البيبلوغرافية
العنوان: The Methods for Reducing the Number of OOVs in Chinese-Uyghur NMT System
المؤلفون: Palidan Tuerxun, Askar Hamdulla, Mewlude Nijat
المصدر: Communications in Computer and Information Science ISBN: 9789813292970
بيانات النشر: Springer Singapore, 2019.
سنة النشر: 2019
مصطلحات موضوعية: Agglutinative language, Vocabulary, Machine translation, business.industry, Mass reduction, Computer science, media_common.quotation_subject, computer.software_genre, Multiple language, Feature (machine learning), Artificial intelligence, business, Declension, computer, Word (computer architecture), Natural language processing, media_common
الوصف: Recently, neural machine translation (NMT) has made significant achievements in multiple language pairs, and surpassed traditional statistical machine translation. However, NMT has strict restrictions on vocabulary, which leads to out-of-vocabulary (OOV) problems. Agglutinative languages, such as Uyghur, which are rich in morphological changes, theoretically have unlimited vocabulary, and we confront with serious OOV problems with these languages in NMT. In a quest of how to reduce OOVs in Chinese - Uyghur pair NMT, we present two different solutions on this study: In the first solution, with regard of the key feature of agglutinative languages, the declension, we segment Uyghur words into stems and affixes. The NMT test we ran on the stem-affix segmented data showed that the number of OOVs reduced from an original 1526 to only 121. In the second solution, we ran a Similar Word Replacement test on low-frequency words from Chinese corpus after training and achieved an even more reduced OOV result of 98. The mass reduction of OOVs from 1.5 thousand to only a hundred signifies the effectiveness of the solutions in this study.
ردمك: 978-981-329-297-0
URL الوصول: https://explore.openaire.eu/search/publication?articleId=doi_________::64d0e30535430eb18782030f37435490
https://doi.org/10.1007/978-981-32-9298-7_15
حقوق: CLOSED
رقم الأكسشن: edsair.doi...........64d0e30535430eb18782030f37435490
قاعدة البيانات: OpenAIRE