تقرير
Multimodal Analysis Of Google Bard And GPT-Vision: Experiments In Visual Reasoning
العنوان: | Multimodal Analysis Of Google Bard And GPT-Vision: Experiments In Visual Reasoning |
---|---|
المؤلفون: | Noever, David, Noever, Samantha Elizabeth Miller |
سنة النشر: | 2023 |
المجموعة: | Computer Science |
مصطلحات موضوعية: | Computer Science - Computer Vision and Pattern Recognition, Computer Science - Computation and Language, Computer Science - Machine Learning |
الوصف: | Addressing the gap in understanding visual comprehension in Large Language Models (LLMs), we designed a challenge-response study, subjecting Google Bard and GPT-Vision to 64 visual tasks, spanning categories like "Visual Situational Reasoning" and "Next Scene Prediction." Previous models, such as GPT4, leaned heavily on optical character recognition tools like Tesseract, whereas Bard and GPT-Vision, akin to Google Lens and Visual API, employ deep learning techniques for visual text recognition. However, our findings spotlight both vision-language model's limitations: while proficient in solving visual CAPTCHAs that stump ChatGPT alone, it falters in recreating visual elements like ASCII art or analyzing Tic Tac Toe grids, suggesting an over-reliance on educated visual guesses. The prediction problem based on visual inputs appears particularly challenging with no common-sense guesses for next-scene forecasting based on current "next-token" multimodal models. This study provides experimental insights into the current capacities and areas for improvement in multimodal LLMs. |
نوع الوثيقة: | Working Paper |
URL الوصول: | http://arxiv.org/abs/2309.16705 |
رقم الأكسشن: | edsarx.2309.16705 |
قاعدة البيانات: | arXiv |
كن أول من يترك تعليقا!