تقرير
VSA4VQA: Scaling a Vector Symbolic Architecture to Visual Question Answering on Natural Images
| العنوان: | VSA4VQA: Scaling a Vector Symbolic Architecture to Visual Question Answering on Natural Images |
|---|---|
| المؤلفون: | Penzkofer, Anna, Shi, Lei, Bulling, Andreas |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence |
| الوصف: | While Vector Symbolic Architectures (VSAs) are promising for modelling spatial cognition, their application is currently limited to artificially generated images and simple spatial queries. We propose VSA4VQA - a novel 4D implementation of VSAs that implements a mental representation of natural images for the challenging task of Visual Question Answering (VQA). VSA4VQA is the first model to scale a VSA to complex spatial queries. Our method is based on the Semantic Pointer Architecture (SPA) to encode objects in a hyperdimensional vector space. To encode natural images, we extend the SPA to include dimensions for object's width and height in addition to their spatial location. To perform spatial queries we further introduce learned spatial query masks and integrate a pre-trained vision-language model for answering attribute-related questions. We evaluate our method on the GQA benchmark dataset and show that it can effectively encode natural images, achieving competitive performance to state-of-the-art deep learning methods for zero-shot VQA. Comment: To be published in the Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci'24) |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2405.03852 |
| رقم الأكسشن: | edsarx.2405.03852 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |