Accelerating the Global Arrays ComEx Runtime using Multiple Progress Ranks

Abstract

Partitioned Global Address Space (PGAS) models are part of system software that are being designed to support communication runtime for exascale applications. Suitability of using MPI as a scalable PGAS communication subsystem has been shown as a viable option due to its standardization and higher performance. We used MPI two-sided semantics with a combination of automatic and user defined splitting of MPI communicators to achieve asynchronous progress. Our implementation can make use of multiple asynchronous progress ranks (PR) per node that can be mapped to the computing architecture of a node in a distributed cluster. We are able to show significant speed up of over 2.0X and scaling of communication bound computational chemistry application over 1024 nodes of state- of-the-art HPC clusters. Our results show that while running a communication bound application workload on a certain number of cluster nodes, a use of optimum number of ranks dedicated for communication may be used to achieve asynchronous communication progress and obtain highest performance.