تقرير
What do we learn from a large-scale study of pre-trained visual representations in sim and real environments?
| العنوان: | What do we learn from a large-scale study of pre-trained visual representations in sim and real environments? |
|---|---|
| المؤلفون: | Silwal, Sneha, Yadav, Karmesh, Wu, Tingfan, Vakil, Jay, Majumdar, Arjun, Arnaud, Sergio, Chen, Claire, Berges, Vincent-Pierre, Batra, Dhruv, Rajeswaran, Aravind, Kalakrishnan, Mrinal, Meier, Franziska, Maksymets, Oleksandr |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Robotics, Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, 68T45 (Primary) 68T40, 68T05(Secondary), I.2.9, I.2.6, I.4.8, I.5.4 |
| الوصف: | We present a large empirical investigation on the use of pre-trained visual representations (PVRs) for training downstream policies that execute real-world tasks. Our study involves five different PVRs, each trained for five distinct manipulation or indoor navigation tasks. We performed this evaluation using three different robots and two different policy learning paradigms. From this effort, we can arrive at three insights: 1) the performance trends of PVRs in the simulation are generally indicative of their trends in the real world, 2) the use of PVRs enables a first-of-its-kind result with indoor ImageNav (zero-shot transfer to a held-out scene in the real world), and 3) the benefits from variations in PVRs, primarily data-augmentation and fine-tuning, also transfer to the real-world performance. See project website for additional details and visuals. Comment: Project website https://pvrs-sim2real.github.io/ |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2310.02219 |
| رقم الأكسشن: | edsarx.2310.02219 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |