تقرير
Efficient All-to-All Collective Communication Schedules for Direct-Connect Topologies
| العنوان: | Efficient All-to-All Collective Communication Schedules for Direct-Connect Topologies |
|---|---|
| المؤلفون: | Basu, Prithwish, Zhao, Liangyu, Fantl, Jason, Pal, Siddharth, Krishnamurthy, Arvind, Khoury, Joud |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Networking and Internet Architecture |
| الوصف: | The all-to-all collective communications primitive is widely used in machine learning (ML) and high performance computing (HPC) workloads, and optimizing its performance is of interest to both ML and HPC communities. All-to-all is a particularly challenging workload that can severely strain the underlying interconnect bandwidth at scale. This paper takes a holistic approach to optimize the performance of all-to-all collective communications on supercomputer-scale direct-connect interconnects. We address several algorithmic and practical challenges in developing efficient and bandwidth-optimal all-to-all schedules for any topology and lowering the schedules to various runtimes and interconnect technologies. We also propose a novel topology that delivers near-optimal all-to-all performance. Comment: HPDC '24 |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2309.13541 |
| رقم الأكسشن: | edsarx.2309.13541 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |