دورية أكاديمية
A novel dual-modal emotion recognition algorithm with fusing hybrid features of audio signal and speech context
| العنوان: | A novel dual-modal emotion recognition algorithm with fusing hybrid features of audio signal and speech context |
|---|---|
| المؤلفون: | Yurui Xu, Hang Su, Guijin Ma, Xiaorui Liu |
| المصدر: | Complex & Intelligent Systems, Vol 9, Iss 1, Pp 951-963 (2022) |
| بيانات النشر: | Springer, 2022. |
| سنة النشر: | 2022 |
| المجموعة: | LCC:Electronic computers. Computer science LCC:Information technology |
| مصطلحات موضوعية: | Emotion recognition, Dual-modal, Pconv, BLSTM, Electronic computers. Computer science, QA75.5-76.95, Information technology, T58.5-58.64 |
| الوصف: | Abstract With regard to human–machine interaction, accurate emotion recognition is a challenging problem. In this paper, efforts were taken to explore the possibility to complete the feature abstraction and fusion by the homogeneous network component, and propose a dual-modal emotion recognition framework that is composed of a parallel convolution (Pconv) module and attention-based bidirectional long short-term memory (BLSTM) module. The Pconv module employs parallel methods to extract multidimensional social features and provides more effective representation capacity. Attention-based BLSTM module is utilized to strengthen key information extraction and maintain the relevance between information. Experiments conducted on the CH-SIMS dataset indicate that the recognition accuracy reaches 74.70% on audio data and 77.13% on text, while the accuracy of the dual-modal fusion model reaches 90.02%. Through experiments it proves the feasibility to process heterogeneous information within homogeneous network component, and demonstrates that attention-based BLSTM module would achieve best coordination with the feature fusion realized by Pconv module. This can give great flexibility for the modality expansion and architecture design. |
| نوع الوثيقة: | article |
| وصف الملف: | electronic resource |
| اللغة: | English |
| تدمد: | 2199-4536 2198-6053 |
| Relation: | https://doaj.org/toc/2199-4536; https://doaj.org/toc/2198-6053 |
| DOI: | 10.1007/s40747-022-00841-3 |
| URL الوصول: | https://doaj.org/article/59d846430821493ca361a5e26e92c1d6 |
| رقم الأكسشن: | edsdoj.59d846430821493ca361a5e26e92c1d6 |
| قاعدة البيانات: | Directory of Open Access Journals |
Find this article in full text from Springer Verlag. 
Full Text Finder | تدمد: | 21994536 21986053 |
|---|---|
| DOI: | 10.1007/s40747-022-00841-3 |