Softwarized Federations of Science Instruments with Edge-Continuum Containers
Significant expansion of capabilities of DOE science complex of supercomputers, instruments and networks is expected as powerful experimental facilities, exascale computers and terabit networks are added. Combined with the advances in edge and cloud computing technologies, DOE science users now have the promise of unprecedented execution of complex, continuum workflows, namely, small and latency-sensitive computations at the edge and site, massive computations at remote HPC systems, and everything in between on the cloud. But, bringing this capability to the science user requires overcoming the overwhelming complexity of forming the federations of systems, and efficiently and effectively orchestrating the workflows while ensuring high utilization of the expensive facilities. Current manual configuration of the federated systems simply will not scale, since the coordination across sites may take weeks to months, often leading to under-utilized and hard-to-diagnose compositions. A powerful, composable software stack will be developed to (i) wrap the systems so that federations can be composed fast in software, and (ii) containerize computations to be orchestrated across the edge, site, cloud and HPC resources.