دورية أكاديمية
A Modular Vision Language Navigation and Manipulation Framework for Long Horizon Compositional Tasks in Indoor Environment.
| العنوان: | A Modular Vision Language Navigation and Manipulation Framework for Long Horizon Compositional Tasks in Indoor Environment. |
|---|---|
| المؤلفون: | Saha H; Department of Mechanical Engineering, Iowa State University, Ames, IA, United States.; Department of Computer Science, Iowa State University, Ames, IA, United States., Fotouhi F; Department of Mechanical Engineering, Iowa State University, Ames, IA, United States.; Department of Computer Science, Iowa State University, Ames, IA, United States., Liu Q; Department of Mechanical Engineering, Iowa State University, Ames, IA, United States., Sarkar S; Department of Mechanical Engineering, Iowa State University, Ames, IA, United States.; Department of Computer Science, Iowa State University, Ames, IA, United States. |
| المصدر: | Frontiers in robotics and AI [Front Robot AI] 2022 Jul 13; Vol. 9, pp. 930486. Date of Electronic Publication: 2022 Jul 13 (Print Publication: 2022). |
| نوع المنشور: | Journal Article |
| اللغة: | English |
| بيانات الدورية: | Publisher: Frontiers Media SA Country of Publication: Switzerland NLM ID: 101749350 Publication Model: eCollection Cited Medium: Internet ISSN: 2296-9144 (Electronic) Linking ISSN: 22969144 NLM ISO Abbreviation: Front Robot AI Subsets: PubMed not MEDLINE |
| أسماء مطبوعة: | Original Publication: Lausanne, Switzerland : Frontiers Media SA, [2014]- |
| مستخلص: | In this paper we propose a new framework-MoViLan (Modular Vision and Language) for execution of visually grounded natural language instructions for day to day indoor household tasks. While several data-driven, end-to-end learning frameworks have been proposed for targeted navigation tasks based on the vision and language modalities, performance on recent benchmark data sets revealed the gap in developing comprehensive techniques for long horizon, compositional tasks (involving manipulation and navigation) with diverse object categories, realistic instructions and visual scenarios with non reversible state changes. We propose a modular approach to deal with the combined navigation and object interaction problem without the need for strictly aligned vision and language training data (e.g., in the form of expert demonstrated trajectories). Such an approach is a significant departure from the traditional end-to-end techniques in this space and allows for a more tractable training process with separate vision and language data sets. Specifically, we propose a novel geometry-aware mapping technique for cluttered indoor environments, and a language understanding model generalized for household instruction following. We demonstrate a significant increase in success rates for long horizon, compositional tasks over recent works on the recently released benchmark data set -ALFRED. Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. (Copyright © 2022 Saha, Fotouhi, Liu and Sarkar.) |
| فهرسة مساهمة: | Keywords: ALFRED; BERT; NLP language parser; depth and mask estimation; graph convolutional mapping; long horizon compositional tasks; robot navigation and manipulation; vision and language module |
| تواريخ الأحداث: | Date Created: 20220804 Latest Revision: 20220804 |
| رمز التحديث: | 20240628 |
| مُعرف محوري في PubMed: | PMC9340572 |
| DOI: | 10.3389/frobt.2022.930486 |
| PMID: | 35923304 |
| قاعدة البيانات: | MEDLINE |
Full Text Finder | تدمد: | 2296-9144 |
|---|---|
| DOI: | 10.3389/frobt.2022.930486 |