تقرير
Reward Tweaking: Maximizing the Total Reward While Planning for Short Horizons
| العنوان: | Reward Tweaking: Maximizing the Total Reward While Planning for Short Horizons |
|---|---|
| المؤلفون: | Tessler, Chen, Mannor, Shie |
| سنة النشر: | 2020 |
| المجموعة: | Computer Science Statistics |
| مصطلحات موضوعية: | Computer Science - Machine Learning, Statistics - Machine Learning |
| الوصف: | In reinforcement learning, the discount factor $\gamma$ controls the agent's effective planning horizon. Traditionally, this parameter was considered part of the MDP; however, as deep reinforcement learning algorithms tend to become unstable when the effective planning horizon is long, recent works refer to $\gamma$ as a hyper-parameter -- thus changing the underlying MDP and potentially leading the agent towards sub-optimal behavior on the original task. In this work, we introduce \emph{reward tweaking}. Reward tweaking learns a surrogate reward function $\tilde r$ for the discounted setting that induces optimal behavior on the original finite-horizon total reward task. Theoretically, we show that there exists a surrogate reward that leads to optimality in the original task and discuss the robustness of our approach. Additionally, we perform experiments in high-dimensional continuous control tasks and show that reward tweaking guides the agent towards better long-horizon returns although it plans for short horizons. Comment: Preprint |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2002.03327 |
| رقم الأكسشن: | edsarx.2002.03327 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |