تقرير
Kinematics-Guided Reinforcement Learning for Object-Aware 3D Ego-Pose Estimation
| العنوان: | Kinematics-Guided Reinforcement Learning for Object-Aware 3D Ego-Pose Estimation |
|---|---|
| المؤلفون: | Luo, Zhengyi, Hachiuma, Ryo, Yuan, Ye, Iwase, Shun, Kitani, Kris M. |
| سنة النشر: | 2020 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computer Vision and Pattern Recognition |
| الوصف: | We propose a method for incorporating object interaction and human body dynamics into the task of 3D ego-pose estimation using a head-mounted camera. We use a kinematics model of the human body to represent the entire range of human motion, and a dynamics model of the body to interact with objects inside a physics simulator. By bringing together object modeling, kinematics modeling, and dynamics modeling in a reinforcement learning (RL) framework, we enable object-aware 3D ego-pose estimation. We devise several representational innovations through the design of the state and action space to incorporate 3D scene context and improve pose estimation quality. We also construct a fine-tuning step to correct the drift and refine the estimated human-object interaction. This is the first work to estimate a physically valid 3D full-body interaction sequence with objects (e.g., chairs, boxes, obstacles) from egocentric videos. Experiments with both controlled and in-the-wild settings show that our method can successfully extract an object-conditioned 3D ego-pose sequence that is consistent with the laws of physics. Comment: Project website: https://zhengyiluo.github.io/projects/contextegopose/ |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2011.04837 |
| رقم الأكسشن: | edsarx.2011.04837 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |