تقرير
Inconsistency-Aware Cross-Attention for Audio-Visual Fusion in Dimensional Emotion Recognition
| العنوان: | Inconsistency-Aware Cross-Attention for Audio-Visual Fusion in Dimensional Emotion Recognition |
|---|---|
| المؤلفون: | Rajasekhar, G, Alam, Jahangir |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computer Vision and Pattern Recognition |
| الوصف: | Leveraging complementary relationships across modalities has recently drawn a lot of attention in multimodal emotion recognition. Most of the existing approaches explored cross-attention to capture the complementary relationships across the modalities. However, the modalities may also exhibit weak complementary relationships, which may deteriorate the cross-attended features, resulting in poor multimodal feature representations. To address this problem, we propose Inconsistency-Aware Cross-Attention (IACA), which can adaptively select the most relevant features on-the-fly based on the strong or weak complementary relationships across audio and visual modalities. Specifically, we design a two-stage gating mechanism that can adaptively select the appropriate relevant features to deal with weak complementary relationships. Extensive experiments are conducted on the challenging Aff-Wild2 dataset to show the robustness of the proposed model. Comment: arXiv admin note: substantial text overlap with arXiv:2403.19554 |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2405.12853 |
| رقم الأكسشن: | edsarx.2405.12853 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |