دورية أكاديمية
Leveraging explainability for understanding object descriptions in ambiguous 3D environments
| العنوان: | Leveraging explainability for understanding object descriptions in ambiguous 3D environments |
|---|---|
| المؤلفون: | Fethiye Irmak Doğan, Gaspar I. Melsión, Iolanda Leite |
| المصدر: | Frontiers in Robotics and AI, Vol 9 (2023) |
| بيانات النشر: | Frontiers Media S.A., 2023. |
| سنة النشر: | 2023 |
| المجموعة: | LCC:Mechanical engineering and machinery LCC:Electronic computers. Computer science |
| مصطلحات موضوعية: | explainability, resolving ambiguities, depth, referring expression comprehension (REC), real-world environments, Mechanical engineering and machinery, TJ1-1570, Electronic computers. Computer science, QA75.5-76.95 |
| الوصف: | For effective human-robot collaboration, it is crucial for robots to understand requests from users perceiving the three-dimensional space and ask reasonable follow-up questions when there are ambiguities. While comprehending the users’ object descriptions in the requests, existing studies have focused on this challenge for limited object categories that can be detected or localized with existing object detection and localization modules. Further, they have mostly focused on comprehending the object descriptions using flat RGB images without considering the depth dimension. On the other hand, in the wild, it is impossible to limit the object categories that can be encountered during the interaction, and 3-dimensional space perception that includes depth information is fundamental in successful task completion. To understand described objects and resolve ambiguities in the wild, for the first time, we suggest a method leveraging explainability. Our method focuses on the active areas of an RGB scene to find the described objects without putting the previous constraints on object categories and natural language instructions. We further improve our method to identify the described objects considering depth dimension. We evaluate our method in varied real-world images and observe that the regions suggested by our method can help resolve ambiguities. When we compare our method with a state-of-the-art baseline, we show that our method performs better in scenes with ambiguous objects which cannot be recognized by existing object detectors. We also show that using depth features significantly improves performance in scenes where depth data is critical to disambiguate the objects and across our evaluation dataset that contains objects that can be specified with and without the depth dimension. |
| نوع الوثيقة: | article |
| وصف الملف: | electronic resource |
| اللغة: | English |
| تدمد: | 2296-9144 |
| Relation: | https://www.frontiersin.org/articles/10.3389/frobt.2022.937772/full; https://doaj.org/toc/2296-9144 |
| DOI: | 10.3389/frobt.2022.937772 |
| URL الوصول: | https://doaj.org/article/2829c8b3155245a38ac511aa9a7bef2a |
| رقم الأكسشن: | edsdoj.2829c8b3155245a38ac511aa9a7bef2a |
| قاعدة البيانات: | Directory of Open Access Journals |
Full Text Finder | تدمد: | 22969144 |
|---|---|
| DOI: | 10.3389/frobt.2022.937772 |