تقرير
Backward error analysis and the qualitative behaviour of stochastic optimization algorithms: Application to stochastic coordinate descent
| العنوان: | Backward error analysis and the qualitative behaviour of stochastic optimization algorithms: Application to stochastic coordinate descent |
|---|---|
| المؤلفون: | Di Giovacchino, Stefano, Higham, Desmond J., Zygalakis, Konstantinos |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science Mathematics Statistics |
| مصطلحات موضوعية: | Mathematics - Optimization and Control, Mathematics - Numerical Analysis, Statistics - Machine Learning |
| الوصف: | Stochastic optimization methods have been hugely successful in making large-scale optimization problems feasible when computing the full gradient is computationally prohibitive. Using the theory of modified equations for numerical integrators, we propose a class of stochastic differential equations that approximate the dynamics of general stochastic optimization methods more closely than the original gradient flow. Analyzing a modified stochastic differential equation can reveal qualitative insights about the associated optimization method. Here, we study mean-square stability of the modified equation in the case of stochastic coordinate descent. Comment: 15 pages; 3 figures |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2309.02082 |
| رقم الأكسشن: | edsarx.2309.02082 |
| قاعدة البيانات: | arXiv |
كن أول من يترك تعليقا!