Student-Initiated Action Advising via Advice Novelty
| العنوان: | Student-Initiated Action Advising via Advice Novelty |
|---|---|
| المؤلفون: | Jeremy Gow, Ercument Ilhan, Diego Perez |
| المصدر: | IEEE Transactions on Games. 14:522-532 |
| بيانات النشر: | Institute of Electrical and Electronics Engineers (IEEE), 2022. |
| سنة النشر: | 2022 |
| مصطلحات موضوعية: | FOS: Computer and information sciences, Computer Science - Machine Learning, Medical education, Novelty, Machine Learning (stat.ML), Machine Learning (cs.LG), Advice (programming), Action (philosophy), Statistics - Machine Learning, Artificial Intelligence, Control and Systems Engineering, ComputingMilieux_COMPUTERSANDEDUCATION, Electrical and Electronic Engineering, Psychology, Software |
| الوصف: | Action advising is a budget-constrained knowledge exchange mechanism between teacher-student peers that can help tackle exploration and sample inefficiency problems in deep reinforcement learning (RL). Most recently, student-initiated techniques that utilise state novelty and uncertainty estimations have obtained promising results. However, the approaches built on these estimations have some potential weaknesses. First, they assume that the convergence of the student's RL model implies less need for advice. This can be misleading in scenarios with teacher absence early on where the student is likely to learn suboptimally by itself; yet also ignore the teacher's assistance later. Secondly, the delays between encountering states and having them to take effect in the RL model updates in presence of the experience replay dynamics cause a feedback lag in what the student actually needs advice for. We propose a student-initiated algorithm that alleviates these by employing Random Network Distillation (RND) to measure the novelty of a piece of advice. Furthermore, we perform RND updates only for the advised states to ensure that the student's own learning does not impair its ability to leverage the teacher. Experiments in GridWorld and MinAtar show that our approach performs on par with the state-of-the-art and demonstrates significant advantages in the scenarios where the existing methods are prone to fail. |
| تدمد: | 2475-1510 2475-1502 |
| URL الوصول: | https://explore.openaire.eu/search/publication?articleId=doi_dedup___::f5799cfa4f5428d889fe91838a21d0f5 https://doi.org/10.1109/tg.2021.3113644 |
| حقوق: | OPEN |
| رقم الأكسشن: | edsair.doi.dedup.....f5799cfa4f5428d889fe91838a21d0f5 |
| قاعدة البيانات: | OpenAIRE |
| تدمد: | 24751510 24751502 |
|---|