Divergence of the ADAM algorithm with fixed-stepsize: a (very) simple example

التفاصيل البيبلوغرافية
العنوان: Divergence of the ADAM algorithm with fixed-stepsize: a (very) simple example
المؤلفون: Toint, Ph. L.
سنة النشر: 2023
المجموعة: Computer Science
مصطلحات موضوعية: Computer Science - Machine Learning, 65K10, 90C26, 90C30, G.6.1, I.2.6
الوصف: A very simple unidimensional function with Lipschitz continuous gradient is constructed such that the ADAM algorithm with constant stepsize, started from the origin, diverges when applied to minimize this function in the absence of noise on the gradient. Divergence occurs irrespective of the choice of the method parameters.
نوع الوثيقة: Working Paper
URL الوصول: http://arxiv.org/abs/2308.00720
رقم الأكسشن: edsarx.2308.00720
قاعدة البيانات: arXiv