تقرير
Safe Wasserstein Constrained Deep Q-Learning
| العنوان: | Safe Wasserstein Constrained Deep Q-Learning |
|---|---|
| المؤلفون: | Kandel, Aaron, Moura, Scott J. |
| سنة النشر: | 2020 |
| المجموعة: | Computer Science Statistics |
| مصطلحات موضوعية: | Computer Science - Machine Learning, Statistics - Machine Learning |
| الوصف: | This paper presents a distributionally robust Q-Learning algorithm (DrQ) which leverages Wasserstein ambiguity sets to provide idealistic probabilistic out-of-sample safety guarantees during online learning. First, we follow past work by separating the constraint functions from the principal objective to create a hierarchy of machines which estimate the feasible state-action space within the constrained Markov decision process (CMDP). DrQ works within this framework by augmenting constraint costs with tightening offset variables obtained through Wasserstein distributionally robust optimization (DRO). These offset variables correspond to worst-case distributions of modeling error characterized by the TD-errors of the constraint Q-functions. This procedure allows us to safely approach the nominal constraint boundaries. Using a case study of lithium-ion battery fast charging, we explore how idealistic safety guarantees translate to generally improved safety relative to conventional methods. |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2002.03016 |
| رقم الأكسشن: | edsarx.2002.03016 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |