Aspect-Aware Response Generation for Multimodal Dialogue System
| العنوان: | Aspect-Aware Response Generation for Multimodal Dialogue System |
|---|---|
| المؤلفون: | Mauajama Firdaus, Asif Ekbal, Nidhi Thakur |
| المصدر: | ACM Transactions on Intelligent Systems and Technology. 12:1-33 |
| بيانات النشر: | Association for Computing Machinery (ACM), 2021. |
| سنة النشر: | 2021 |
| مصطلحات موضوعية: | Service (systems architecture), Computer science, 02 engineering and technology, computer.software_genre, Theoretical Computer Science, Bridging (programming), Multimodality, Task (project management), Artificial Intelligence, Human–computer interaction, 020204 information systems, 0202 electrical engineering, electronic engineering, information engineering, 020201 artificial intelligence & image processing, Product (category theory), Dialog box, Dialog system, Baseline (configuration management), computer |
| الوصف: | Multimodality in dialogue systems has opened up new frontiers for the creation of robust conversational agents. Any multimodal system aims at bridging the gap between language and vision by leveraging diverse and often complementary information from image, audio, and video, as well as text. For every task-oriented dialog system, different aspects of the product or service are crucial for satisfying the user’s demands. Based upon the aspect, the user decides upon selecting the product or service. The ability to generate responses with the specified aspects in a goal-oriented dialogue setup facilitates user satisfaction by fulfilling the user’s goals. Therefore, in our current work, we propose the task of aspect controlled response generation in a multimodal task-oriented dialog system. We employ a multimodal hierarchical memory network for generating responses that utilize information from both text and images. As there was no readily available data for building such multimodal systems, we create a Multi-Domain Multi-Modal Dialog (MDMMD++) dataset. The dataset comprises the conversations having both text and images belonging to the four different domains, such as hotels, restaurants, electronics, and furniture. Quantitative and qualitative analysis on the newly created MDMMD++ dataset shows that the proposed methodology outperforms the baseline models for the proposed task of aspect controlled response generation. |
| تدمد: | 2157-6912 2157-6904 |
| URL الوصول: | https://explore.openaire.eu/search/publication?articleId=doi_________::9e935b138d427bf5251aef3fa058697d https://doi.org/10.1145/3430752 |
| رقم الأكسشن: | edsair.doi...........9e935b138d427bf5251aef3fa058697d |
| قاعدة البيانات: | OpenAIRE |
| تدمد: | 21576912 21576904 |
|---|