تقرير
Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers
| العنوان: | Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers |
|---|---|
| المؤلفون: | Bassetti, Federico, Gherardi, Marco, Ingrosso, Alessandro, Pastore, Mauro, Rotondo, Pietro |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science Mathematics Condensed Matter Statistics |
| مصطلحات موضوعية: | Statistics - Machine Learning, Condensed Matter - Disordered Systems and Neural Networks, Computer Science - Machine Learning, Mathematics - Statistics Theory, 62E20, 62E15, 82B44 |
| الوصف: | Deep linear networks have been extensively studied, as they provide simplified models of deep learning. However, little is known in the case of finite-width architectures with multiple outputs and convolutional layers. In this manuscript, we provide rigorous results for the statistics of functions implemented by the aforementioned class of networks, thus moving closer to a complete characterization of feature learning in the Bayesian setting. Our results include: (i) an exact and elementary non-asymptotic integral representation for the joint prior distribution over the outputs, given in terms of a mixture of Gaussians; (ii) an analytical formula for the posterior distribution in the case of squared error loss function (Gaussian likelihood); (iii) a quantitative description of the feature learning infinite-width regime, using large deviation theory. From a physical perspective, deep architectures with multiple outputs or convolutional layers represent different manifestations of kernel shape renormalization, and our work provides a dictionary that translates this physics intuition and terminology into rigorous Bayesian statistics. |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2406.03260 |
| رقم الأكسشن: | edsarx.2406.03260 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |