A Variance-Reduced Aggregation Based Gradient Tracking method for Distributed Optimization over Directed Networks

التفاصيل البيبلوغرافية
العنوان: A Variance-Reduced Aggregation Based Gradient Tracking method for Distributed Optimization over Directed Networks
المؤلفون: Zhao, Shengchao, Song, Siyuan, Liu, Yongchao
سنة النشر: 2023
المجموعة: Mathematics
مصطلحات موضوعية: Mathematics - Optimization and Control
الوصف: This paper studies the distributed optimization problem over directed networks with noisy information-sharing. To resolve the imperfect communication issue over directed networks, a series of noise-robust variants of Push-Pull/AB method have been developed. These methods improve the robustness of Push-Pull method against the information-sharing noise through adding small factors on weight matrices and replacing the global gradient tracking with the cumulative gradient tracking. Based on the two techniques, we propose a new variant of the Push-Pull method by presenting a novel mechanism of inter-agent information aggregation, named variance-reduced aggregation (VRA). VRA helps us to release some conditions on the objective function and networks. When the objective function is convex and the sharing-information noise is variance-unbounded, it can be shown that the proposed method converges to the optimal solution almost surely. When the objective function is strongly convex and the sharing-information noise is variance-bounded, the proposed method achieves the convergence rate of $\mathcal{O}\left(k^{-(1-\epsilon)}\right)$ in the mean square sense, where $\epsilon$ could be close to 0 infinitely. Simulated experiments on ridge regression problems verify the effectiveness of the proposed method.
نوع الوثيقة: Working Paper
URL الوصول: http://arxiv.org/abs/2307.14776
رقم الأكسشن: edsarx.2307.14776
قاعدة البيانات: arXiv