تقرير
On approximating dropout noise injection
العنوان: | On approximating dropout noise injection |
---|---|
المؤلفون: | Schluter, Natalie |
سنة النشر: | 2019 |
المجموعة: | Computer Science Statistics |
مصطلحات موضوعية: | Computer Science - Machine Learning, Statistics - Machine Learning |
الوصف: | This paper examines the assumptions of the derived equivalence between dropout noise injection and $L_2$ regularisation for logistic regression with negative log loss. We show that the approximation method is based on a divergent Taylor expansion, making, subsequent work using this approximation to compare the dropout trained logistic regression model with standard regularisers unfortunately ill-founded to date. Moreover, the approximation approach is shown to be invalid using any robust constraints. We show how this finding extends to general neural network topologies that use a cross-entropy prediction layer. Comment: Submitted to NeurIPS 2019 |
نوع الوثيقة: | Working Paper |
URL الوصول: | http://arxiv.org/abs/1905.11320 |
رقم الأكسشن: | edsarx.1905.11320 |
قاعدة البيانات: | arXiv |
الوصف غير متاح. |