تقرير
Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension
| العنوان: | Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension |
|---|---|
| المؤلفون: | Karhadkar, Kedar, Murray, Michael, Montúfar, Guido |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science Statistics |
| مصطلحات موضوعية: | Statistics - Machine Learning, Computer Science - Machine Learning |
| الوصف: | Bounds on the smallest eigenvalue of the neural tangent kernel (NTK) are a key ingredient in the analysis of neural network optimization and memorization. However, existing results require distributional assumptions on the data and are limited to a high-dimensional setting, where the input dimension $d_0$ scales at least logarithmically in the number of samples $n$. In this work we remove both of these requirements and instead provide bounds in terms of a measure of the collinearity of the data: notably these bounds hold with high probability even when $d_0$ is held constant versus $n$. We prove our results through a novel application of the hemisphere transform. Comment: 47 pages |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2405.14630 |
| رقم الأكسشن: | edsarx.2405.14630 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |