تقرير
Dimension-free deterministic equivalents for random feature regression
| العنوان: | Dimension-free deterministic equivalents for random feature regression |
|---|---|
| المؤلفون: | Defilippis, Leonardo, Loureiro, Bruno, Misiakiewicz, Theodor |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science Condensed Matter Statistics |
| مصطلحات موضوعية: | Statistics - Machine Learning, Condensed Matter - Disordered Systems and Neural Networks, Computer Science - Machine Learning |
| الوصف: | In this work we investigate the generalization performance of random feature ridge regression (RFRR). Our main contribution is a general deterministic equivalent for the test error of RFRR. Specifically, under a certain concentration property, we show that the test error is well approximated by a closed-form expression that only depends on the feature map eigenvalues. Notably, our approximation guarantee is non-asymptotic, multiplicative, and independent of the feature map dimension -- allowing for infinite-dimensional features. We expect this deterministic equivalent to hold broadly beyond our theoretical analysis, and we empirically validate its predictions on various real and synthetic datasets. As an application, we derive sharp excess error rates under standard power-law assumptions of the spectrum and target decay. In particular, we provide a tight result for the smallest number of features achieving optimal minimax error rate. |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2405.15699 |
| رقم الأكسشن: | edsarx.2405.15699 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |