تقرير
Asymptotic Analysis of Deep Residual Networks
| العنوان: | Asymptotic Analysis of Deep Residual Networks |
|---|---|
| المؤلفون: | Cont, Rama, Rossier, Alain, Xu, Renyuan |
| سنة النشر: | 2022 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Machine Learning, 60F17, 60F25, 68T05 |
| الوصف: | We investigate the asymptotic properties of deep Residual networks (ResNets) as the number of layers increases. We first show the existence of scaling regimes for trained weights markedly different from those implicitly assumed in the neural ODE literature. We study the convergence of the hidden state dynamics in these scaling regimes, showing that one may obtain an ODE, a stochastic differential equation (SDE) or neither of these. In particular, our findings point to the existence of a diffusive regime in which the deep network limit is described by a class of stochastic differential equations (SDEs). Finally, we derive the corresponding scaling limits for the backpropagation dynamics. Comment: 49 pages, 12 figures. arXiv admin note: substantial text overlap with arXiv:2105.12245 |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2212.08199 |
| رقم الأكسشن: | edsarx.2212.08199 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |