Currently, more than 25% of supercomputers employ GPUs due to their massively parallel and power-efficient architectures. However, programming GPUs efficiently in a large scale system is a demanding task not only for computational scientists but also for programming experts as multi-GPU programming requires managing distinct address spaces, generating GPU-specific code and handling inter-device communication. To ease the programming effort, we propose a tiling-based high-level GPU programming model for structured grid problems. The model abstracts data decomposition, memory management and generation of GPU specific code, and hides all types of data transfer overheads. We demonstrate the effectiveness of the programming model on a heat simulation and a real-life cardiac modeling on a single GPU, on a single node with multiple-GPUs and multiple-nodes with multiple-GPUs. We also present performance comparisons under different hardware and software configurations. The results show that the programming model successfully overlaps communication and provides good speedup on 192 GPUs.
