تقرير
How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech
| العنوان: | How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech |
|---|---|
| المؤلفون: | Yedetore, Aditya, Linzen, Tal, Frank, Robert, McCoy, R. Thomas |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computation and Language, J.4, I.2.7 |
| الوصف: | When acquiring syntax, children consistently choose hierarchical rules over competing non-hierarchical possibilities. Is this preference due to a learning bias for hierarchical structure, or due to more general biases that interact with hierarchical cues in children's linguistic input? We explore these possibilities by training LSTMs and Transformers - two types of neural networks without a hierarchical bias - on data similar in quantity and content to children's linguistic input: text from the CHILDES corpus. We then evaluate what these models have learned about English yes/no questions, a phenomenon for which hierarchical structure is crucial. We find that, though they perform well at capturing the surface statistics of child-directed speech (as measured by perplexity), both model types generalize in a way more consistent with an incorrect linear rule than the correct hierarchical rule. These results suggest that human-like generalization from text alone requires stronger biases than the general sequence-processing biases of standard neural network architectures. Comment: 10 pages plus references and appendices; accepted to ACL |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2301.11462 |
| رقم الأكسشن: | edsarx.2301.11462 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |