تقرير
On approximating dropout noise injection
| العنوان: | On approximating dropout noise injection |
|---|---|
| المؤلفون: | Schluter, Natalie |
| سنة النشر: | 2019 |
| المجموعة: | Computer Science Statistics |
| مصطلحات موضوعية: | Computer Science - Machine Learning, Statistics - Machine Learning |
| الوصف: | This paper examines the assumptions of the derived equivalence between dropout noise injection and $L_2$ regularisation for logistic regression with negative log loss. We show that the approximation method is based on a divergent Taylor expansion, making, subsequent work using this approximation to compare the dropout trained logistic regression model with standard regularisers unfortunately ill-founded to date. Moreover, the approximation approach is shown to be invalid using any robust constraints. We show how this finding extends to general neural network topologies that use a cross-entropy prediction layer. Comment: Submitted to NeurIPS 2019 |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/1905.11320 |
| رقم الأكسشن: | edsarx.1905.11320 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |