تقرير
Unsupervised Multimodal Word Discovery based on Double Articulation Analysis with Co-occurrence cues
| العنوان: | Unsupervised Multimodal Word Discovery based on Double Articulation Analysis with Co-occurrence cues |
|---|---|
| المؤلفون: | Taniguchi, Akira, Murakami, Hiroaki, Ozaki, Ryo, Taniguchi, Tadahiro |
| سنة النشر: | 2022 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Robotics |
| الوصف: | Human infants acquire their verbal lexicon with minimal prior knowledge of language based on the statistical properties of phonological distributions and the co-occurrence of other sensory stimuli. This study proposes a novel fully unsupervised learning method for discovering speech units using phonological information as a distributional cue and object information as a co-occurrence cue. The proposed method can acquire words and phonemes from speech signals using unsupervised learning and utilize object information based on multiple modalities-vision, tactile, and auditory-simultaneously. The proposed method is based on the nonparametric Bayesian double articulation analyzer (NPB-DAA) discovering phonemes and words from phonological features, and multimodal latent Dirichlet allocation (MLDA) categorizing multimodal information obtained from objects. In an experiment, the proposed method showed higher word discovery performance than baseline methods. Words that expressed the characteristics of objects (i.e., words corresponding to nouns and adjectives) were segmented accurately. Furthermore, we examined how learning performance is affected by differences in the importance of linguistic information. Increasing the weight of the word modality further improved performance relative to that of the fixed condition. Comment: Accepted to IEEE TRANSACTIONS ON COGNITIVE DEVELOPMENTAL SYSTEMS |
| نوع الوثيقة: | Working Paper |
| DOI: | 10.1109/TCDS.2023.3307555 |
| URL الوصول: | http://arxiv.org/abs/2201.06786 |
| رقم الأكسشن: | edsarx.2201.06786 |
| قاعدة البيانات: | arXiv |
| DOI: | 10.1109/TCDS.2023.3307555 |
|---|