تقرير
Bridging Algorithmic Information Theory and Machine Learning: A New Approach to Kernel Learning
العنوان: | Bridging Algorithmic Information Theory and Machine Learning: A New Approach to Kernel Learning |
---|---|
المؤلفون: | Hamzi, Boumediene, Hutter, Marcus, Owhadi, Houman |
المصدر: | Physica D: Nonlinear Phenomena, 2024, 134153 |
سنة النشر: | 2023 |
المجموعة: | Computer Science Mathematics Statistics |
مصطلحات موضوعية: | Computer Science - Machine Learning, Computer Science - Information Theory, Statistics - Machine Learning |
الوصف: | Machine Learning (ML) and Algorithmic Information Theory (AIT) look at Complexity from different points of view. We explore the interface between AIT and Kernel Methods (that are prevalent in ML) by adopting an AIT perspective on the problem of learning kernels from data, in kernel ridge regression, through the method of Sparse Kernel Flows. In particular, by looking at the differences and commonalities between Minimal Description Length (MDL) and Regularization in Machine Learning (RML), we prove that the method of Sparse Kernel Flows is the natural approach to adopt to learn kernels from data. This approach aligns naturally with the MDL principle, offering a more robust theoretical basis than the existing reliance on cross-validation. The study reveals that deriving Sparse Kernel Flows does not require a statistical approach; instead, one can directly engage with code-lengths and complexities, concepts central to AIT. Thereby, this approach opens the door to reformulating algorithms in machine learning using tools from AIT, with the aim of providing them a more solid theoretical foundation. Comment: An earlier version of this paper appeared at https://www.researchgate.net/publication/371875631_A_note_on_learning_kernels_from_data_from_an_Algorithmic_Information_Theoretic_point_of_view. arXiv admin note: text overlap with arXiv:2111.13037, arXiv:2007.05074 |
نوع الوثيقة: | Working Paper |
DOI: | 10.13140/RG.2.2.36344.01285 |
DOI: | 10.1016/j.physd.2024.134153 |
URL الوصول: | http://arxiv.org/abs/2311.12624 |
رقم الأكسشن: | edsarx.2311.12624 |
قاعدة البيانات: | arXiv |
DOI: | 10.13140/RG.2.2.36344.01285 |
---|