تقرير
Reward-Predictive Clustering
العنوان: | Reward-Predictive Clustering |
---|---|
المؤلفون: | Lehnert, Lucas, Frank, Michael J., Littman, Michael L. |
سنة النشر: | 2022 |
المجموعة: | Computer Science |
مصطلحات موضوعية: | Computer Science - Machine Learning, Computer Science - Artificial Intelligence |
الوصف: | Recent advances in reinforcement-learning research have demonstrated impressive results in building algorithms that can out-perform humans in complex tasks. Nevertheless, creating reinforcement-learning systems that can build abstractions of their experience to accelerate learning in new contexts still remains an active area of research. Previous work showed that reward-predictive state abstractions fulfill this goal, but have only be applied to tabular settings. Here, we provide a clustering algorithm that enables the application of such state abstractions to deep learning settings, providing compressed representations of an agent's inputs that preserve the ability to predict sequences of reward. A convergence theorem and simulations show that the resulting reward-predictive deep network maximally compresses the agent's inputs, significantly speeding up learning in high dimensional visual control tasks. Furthermore, we present different generalization experiments and analyze under which conditions a pre-trained reward-predictive representation network can be re-used without re-training to accelerate learning -- a form of systematic out-of-distribution transfer. |
نوع الوثيقة: | Working Paper |
URL الوصول: | http://arxiv.org/abs/2211.03281 |
رقم الأكسشن: | edsarx.2211.03281 |
قاعدة البيانات: | arXiv |
كن أول من يترك تعليقا!