Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “code portability”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

79 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

To Exascale and Beyond—The Simple Cloud-Resolving E3SM Atmosphere Model (SCREAM), a Performance Portable Global Atmosphere Model for Cloud-Resolving Scales

The new generation of heterogeneous CPU/GPU computer systems offer much greater computational performance but are not yet widely used for climate modeling. One reason for this is that traditional climate models were written before GPUs were available and would require an extensive overhaul to run on these new machines. In addition, even conventional “high–resolution” simulations don't currently provide enough parallel work to keep GPUs busy, so the benefits of such overhaul would be limited for the types of simulations climate scientists are accustomed to. The vision of the Simple Cloud-Resolving Energy Exascale Earth System (E3SM) Atmosphere Model (SCREAM) project is to create a global atmospheric model with the architecture to efficiently use GPUs and horizontal resolution sufficient to fully take advantage of GPU parallelism. After 5 years of model development, SCREAM is finally ready for use. In this paper, we describe the design of this new code, its performance on both CPU and heterogeneous machines, and its ability to simulate real-world climate via a set of four 40 day simulations covering all 4 seasons of the year.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Performance Results on CPU/GPU Exascale Architectures for OMEGA: The Ocean Model for E3SM Global Applications

The US Department of Energy (DOE) conducts climate simulations on some of the world’s largest supercomputers. These exascale machines use heterogeneous architectures with both CPUs and GPUs, and scientific codes must adapt to make full use of this computing power. Los Alamos National Lab is developing Omega: The Ocean Model for E3SM Global Applications, which is specifically designed for modern exascale computers. It uses external libraries that have been optimized for a variety of architectures to run on different supercomputers. Omega is an unstructured-mesh ocean model based on TRiSK numerical methods. It will be the new ocean component of the DOE’s Energy Exascale Earth System Model (E3SM). The algorithms in Omega follow those of the current ocean component, MPAS-Ocean, but it will be written in C++ rather than Fortran to take advantage of the Kokkos performance portability library. Omega spatial operators are written as Kokkos kernels to run efficiently on both CPUs and GPUs. Work on Omega began in 2023 with a new C++ framework for unstructured mesh partitioning, halo exchanges, parallel IO, and Kokkos interfaces. The current version, Omega-0, is being developed to solve the shallow water equations and at present includes all of the tendency terms but not time stepping. Here we share the results of Omega-0 verification and performance testing. Verification includes unit tests implemented with CTest as well as convergence tests in Polaris, an in-house python package with a large suite of test problems. Performance tests compare simulations conducted on CPUs versus GPUs and across different architectures: tests are run on Frontier, which has AMD “Optimized 3rd Gen EPYC” CPUs and AMD MI250X GPUs, as well as Perlmutter, which is composed of AMD EPYC 7763 CPUs and NVIDIA A100 GPUs.

58 GEOSCIENCES↗
DOE OSTI2024

Western States Building Energy & Controls Apprenticeship (BECA) Program

How does this project help to understand the challenges of workforce development in the commercial building energy management industry? The purpose of this project was to create a replicable, scalable, and portable apprenticeship program for building energy management and controls. This paper will demonstrate how local processes can be more adaptable and inclusive than federal processes in achieving workforce development goals. Our work provides insights into the technical effectiveness of the program, enabling others to achieve greater success in their workforce development initiatives. The project required more time and financial resources than initially anticipated and spent a year in a no-cost extension working to accomplish the Statement of Project Objectives. The original goal was to create an Industry Related Apprenticeship Program (IRAP). The development of this apprenticeship presented challenges due to the lack of existing programs for reference and the absence of relevant industry classification codes by the Department of Labor (DOL). Notably, the role of a commercial building energy analyst is not recognized by the DOL. The SIC (industry codes) do not have a good description of this job. There are many that may fall into related categories, but none are the actual duties of an energy analyst. Discussions with the DOL indicated that substantial groundwork was necessary before a national apprenticeship program could be implemented, causing delays in the program’s commencement. As a result, students experienced longer wait times before starting their apprenticeship component. The apprenticeship program officially launched on September 14, 2021. The success of the State of Oregon’s apprenticeship program, the first of its kind in the state, underscored the flexibility and effectiveness of local initiatives compared to federal efforts. The COVID-19 pandemic also significantly impacted the project’s success. Beginning in March 2020, the pandemic led to widespread closures of schools and colleges by fall 2020. By September 2021, when the apprenticeship option became available, enrollment in colleges and universities nationwide had decreased, affecting student participation in the program. Moreover, as employers transitioned their employees to remote work, there was limited interaction with external personnel, influencing the willingness of training agents (employers) to integrate additional workers into their teams. This paper addresses several challenges encountered during the project, with the hope that future workforce development efforts will benefit from these experiences. We Final Technical Report 4 | Page encourage others to engage with state and federal agencies to enhance and update pathways for workforce development and apprenticeship programs.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

autoGEMM: Pushing the Limits of Irregular Matrix Multiplication on Arm Architectures

This paper presents an open-source library that pushes the limits of performance portability for irregular General Matrix Multiplication (GEMM) on the widely-used Arm architectures. Our library, autoGEMM, is designed to support a wide range of Arm processors: from edge devices to HPC-grade CPUs. autoGEMM generates optimized kernels for various hardware configurations by auto-combining fragments of autogenerated micro-kernels that employ hand-written optimizations to maximize computational efficiency. We optimize the kernel pipeline by tuning the register reuse and the data load/store overlapping. In addition, we use a dynamic tiling scheme to generate balanced tile shapes. Finally, we position autoGEMM on top of the TVM framework where our dynamic tiling scheme prunes the search space for TVM to identify the optimal combination of parameters for code optimization. Evaluations on five different classes of Arm chips demonstrate the advantages of autoGEMM. For small matrices, autoGEMM achieves 98% of peak and up to 2.0x speedup over state-of-the-art libraries such as LIBXSMM and LibShalom. For irregular matrices (i.e. tall skinny and long rectangles), autoGEMM is 1.3-2.0x faster than widely-used libraries such as OpenBLAS and Eigen. autoGEMM is publicly available at: https://github.com/wudu98/autoGEMM.

Wu, Du↗
DOE OSTI2025

The Water Table Model (WTM) (v2.0.1): coupled groundwater and dynamic lake modelling

Abstract. Ice-free land comprises 26 % of the Earth's surface and holds liquid water that delineates ecosystems, affects global geochemical cycling, and modulates sea levels. However, we currently lack the capacity to simulate and predict these terrestrial water changes across the full range of relevant spatial (watershed to global) and temporal (monthly to millennial) scales. To address this knowledge gap, we present the Water Table Model (WTM), which integrates coupled components to compute dynamic lake and groundwater levels. The groundwater component solves the 2D horizontal groundwater flow equation using non-linear equation solvers from the C++ PETSc (Portable, Extensible Toolkit for Scientific Computation) library. The dynamic lake component makes use of the Fill–Spill–Merge (FSM) algorithm to move surface water into lakes, where it may evaporate or affect groundwater flow. In a proof-of-concept application, we demonstrate the continental-scale capabilities of the WTM by simulating the steady-state climate-driven water table for the present day and the Last Glacial Maximum (LGM; 21 000 calendar years before present) across the North American continent. During the LGM, North America stored an additional 14.98 cm of sea-level equivalent (SLE) in lakes and groundwater compared to the climate-driven present-day scenario. We compare the present-day result to other simulations and real-world data. Open-source code for the WTM is available on GitHub and Zenodo.

Callaghan, Kerry L. (ORCID:0000000226740838)↗
DOE OSTI2025

Development of a River Dynamical Core for E3SM to simulate compound flooding on Exascale-class heterogeneous supercomputers

Flooding events pose significant risk to human life, property, and infrastructure. Physically-consistent quantification of altered flood risks in global models requires hyper-resolution (~1 km) or fine flood simulations using two-dimensional (2D) physics schemes, both of which are unavailable in the current generation Earth System Models. Here, in this work, we have developed the River Dynamical Core (RDycore), which is an open-source, 2D shallow water equation (SWE) library for the U.S. Department of Energy's Energy Exascale Earth System Model (E3SM). RDycore uses PETSc and libCEED libraries that allows it to run efficiently on CPUs and GPUs, as well as select a time-integration algorithm at runtime without requiring any code modifications. RDycore achieves spatial error convergence rates for problems with analytical and manufactured solutions similar to those reported previously in the literature, or consistent with the implemented first-order spatial discretization scheme. RDycore's accuracy in predicting flooding for a well-studied dam break problem is comparable to existing SWE models. For a problem with 471 million grid cells, RDycore achieves a speedup of 6.6x and 7.6x on GPUs compared to CPUs when using 320 compute nodes on DOE's Perlmutter and Frontier supercomputers, respectively. The one-way coupling of the RDycore library within E3SM is demonstrated by performing multiple 5-day flooding simulations during Hurricane Harvey driven by five precipitation datasets. The E3SM--RDycore simulations at 30 m spatial resolution accurately simulate maximum water height during the hurricane when benchmarked against a previously published study and achieve a speedup of 15x (Perlmutter) and 21x (Frontier) on GPUs relative to CPUs. The work presented here is the foundational step in providing hardware and algorithmic portability framework for simulating kilometer-scale river dynamics within E3SM.

Flood Simulation↗
DOE OSTI2025

Experiences with SYCL on AMD GPUs with Kokkos

With the recent diversification of the hardware landscape in the high-performance computing (HPC) community, performance-portability solutions are becoming more and more important. One of the most popular choices is Kokkos, which recently became a Linux Foundation project. Most of its development is supported by the US Department of Energy and the French Alternative Energies and Atomic Energy Commission. Kokkos is implemented as a C++ library with multiple backends to support CPUs as well as various GPU architectures. These backends include OpenMP, CUDA, HIP, and also SCYL. This approach enables users to leverage the preferred vendor toolchain for the respective platform (e.g. CUDA, ROCm, OneAPI). The SYCL backend is used to target Intel GPUs, in particular to support the Aurora exascale supercomputer. However, SYCL itself also offers a large degree of portability, and in fact Kokkos’ CI for SYCL has been running on NVIDIA hardware due to a lack of access to Intel GPUs. In this report, we describe our experience with using Kokkos SYCL backend on AMD GPUs targeting the Frontier supercomputer at Oak Ridge National Laboratory. The two major SYCL implementations are DPC++ and AdaptiveCpp. While the Kokkos SYCL backend has been implemented using the former, the latter was the first implementation to target AMD GPUs. We will discuss the experience with both of these SYCL implementations in terms of functionality and performance. Using Kokkos to evaluate SYCL toolchains has a number of benefits. Kokkos’ use of SYCL is fairly complex, exercising features such as graphs, relocatable device functions, atomics – including for non-arithmetic types, as well as pinned and page migratable memory allocations. Kokkos also needs to implement capabilities such as Kokkos’ hierarchical parallelism that are not a straight-forward mapping to SYCL capabilities. Furthermore, a large number of libraries and applications that represent diverse use cases are implemented in Kokkos, providing readily available test cases for a toolchain evaluation. Preliminary results show that support for AMD GPUs in DPC++ is much less mature than for NVIDIA GPUs or Intel GPUs. While the situation has improved significantly over the last year, we still encounter many runtime failures, dispatching problems, and code generation issues. With AdaptiveCpp the challenges arise even earlier in the evaluation process. Since Kokkos’ SYCL implementation is largely focused on supporting Intel GPUs, we opted to leverage SYCL extensions which are available in DPC++ but not in AdaptiveCpp. Furthermore, AdaptiveCpp appears to be less conformant with the SYCL2020 standard which Kokkos relies on. In some cases, we are able to work around the lack of feature support, in other cases we have to disable certain Kokkos capabilities to evaluate the toolchain. Our evaluation will leverage Kokkos’ unit tests to establish basic functionality and feature completeness. We then use simple benchmarks for components of a CG implementation as a measure of usability and performance of the SYCL toolchains.

97 MATHEMATICS AND COMPUTING↗