Matrix and Array (MATAR) Library
MATAR is a C++ software library to allow developers to easily create and use dense and sparse data representations that are also portable across disparate architectures using Kokkos.
Morgan, Nathaniel↗
SEARCH · Engineering Papers
Results for “kokkos”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
ORNL/grit
Grit is a software library designed specifically for conducting particle-based Lagrangian simulations with CPU/GPUperformance portability. This library allows researchers and engineers to perform a wide range of simulations, including multiphase flow, and more. Grit employs Message Passing Interface (MPI) for distributed memory parallelism and Kokkos programming model for on-node shared memory parallelism with performance portability across different architectures of GPUs and multi-core/manycore CPUs.
Ge, Wenjun [Oak Ridge National Laboratory (ORNL), ↗
A survey of software implementations used by application codes in the Exascale Computing Project
The US Department of Energy Office of Science and the National Nuclear Security Administration initiated the Exascale Computing Project (ECP) in 2016 to prepare mission-relevant applications and scientific software for the delivery of the exascale computers starting in 2023. The ECP currently supports 24 efforts directed at specific applications and six supporting co-design projects. These 24 application projects contain 62 application codes that are implemented in three high-level languages—C, C++, and Fortran—and use 22 combinations of graphical processing unit programming models. The most common implementation language is C++, which is used in 53 different application codes. The most common programming models across ECP applications are CUDA and Kokkos, which are employed in 15 and 14 applications, respectively. This article provides a survey of the programming languages and models used in the ECP applications codebase that will be used to achieve performance on the future exascale hardware platforms.
97 MATHEMATICS AND COMPUTING↗
Parthenon—a performance portable block-structured adaptive mesh refinement framework
On the path to exascale the landscape of computer device architectures and corresponding programming models has become much more diverse. While various low-level performance portable programming models are available, support at the application level lacks behind. To address this issue, we present the performance portable block-structured adaptive mesh refinement (AMR) framework Parthenon, derived from the well-tested and widely used Athena++ astrophysical magnetohydrodynamics code, but generalized to serve as the foundation for a variety of downstream multi-physics codes. Parthenon adopts the Kokkos programming model, and provides various levels of abstractions from multidimensional variables, to packages defining and separating components, to launching of parallel compute kernels. Parthenon allocates all data in device memory to reduce data movement, supports the logical packing of variables and mesh blocks to reduce kernel launch overhead, and employs one-sided, asynchronous MPI calls to reduce communication overhead in multi-node simulations. Using a hydrodynamics miniapp, we demonstrate weak and strong scaling on various architectures including AMD and NVIDIA GPUs, Intel and AMD x86 CPUs, IBM Power9 CPUs, as well as Fujitsu A64FX CPUs. At the largest scale on Frontier (the first TOP500 exascale machine), the miniapp reaches a total of 1.7 × 10 13 zone-cycles/s on 9216 nodes (73,728 logical GPUs) at [Formula: see text] weak scaling parallel efficiency (starting from a single node). In combination with being an open, collaborative project, this makes Parthenon an ideal framework to target exascale simulations in which the downstream developers can focus on their specific application rather than on the complexity of handling massively-parallel, device-accelerated AMR.
97 MATHEMATICS AND COMPUTING↗
Adamantine 1.0: A Thermomechanical Simulator for Additive Manufacturing
Adamantine is a thermomechanical simulation code that is written in C++ and built on top of deal.II (Arndt et al., 2023), p4est (Burstedde et al., 2011), ArborX (Lebrun-Grandié et al., 2020), Trilinos (The Trilinos Project Team, 2020), and Kokkos (Trott et al., 2022). Adamantine was developed with additive manufacturing in mind and it is particularly well adapted to simulate fused filament fabrication, directed energy deposition, and powder bed fusion. Adamantine employs the finite element method with adaptive mesh refinement to solve a nonlinear anisotropic heat equation, enabling support for various additive manufacturing processes. It can also perform elastoplastic and thermoelastoplastic simulations. It can handle materials in three distinct phases (solid, liquid, and powder) to accurately reflect the physical state during different stages of the manufacturing process. To enhance simulation accuracy, adamantine incorporates data assimilation techniques (Asch et al., 2016). This allows it to integrate experimental data from sensors like thermocouples and infrared (IR) cameras. This combined approach helps account for errors arising from input parameters, material properties, models, and numerical calculations, leading to more realistic simulations that reflect what occurs in a particular print.
36 MATERIALS SCIENCE↗
ECP Report: Update on Proxy Applications and Vendor Interactions
The ExaLearn miniGAN team (Ellis and Rajamanickam) have released miniGAN, a generative adversarial network(GAN) proxy application, through the ECP proxy application suite. miniGAN is the first machine learning proxy application in the suite (note: the ECP CANDLE project did previously release some benchmarks) and models the performance for training generator and discriminator networks. The GAN's generator and discriminator generate plausible 2D/3D maps and identify fake maps, respectively. miniGAN aims to be a proxy application for related applications in cosmology (CosmoFlow, ExaGAN) and wind energy (ExaWind). miniGAN has been developed so that optimized mathematical kernels (e.g., kernels provided by Kokkos Kernels) can be plugged into to the proxy application to explore potential performance improvements. miniGAN has been released as open source software and is available through the ECP proxy application website (https://proxyapps.exascaleproject.ordecp-proxy-appssuite/) and on GitHub (https://github.com/SandiaMLMiniApps/miniGAN). As part of this release, a generator is provided to generate a data set (series of images) that are inputs to the proxy application.
97 MATHEMATICS AND COMPUTING↗
(U) Assessment of new hydrodynamics capabilities in EAP codes to support DSW applications (L2 Milestone Review) [Slides]
The agenda for the presentation is: introduction and overview; method description; Hydrodynamics problems; LED multi-physics problems; HED multi-physics problems; Kokkos port for ATS-1 and ATS-2; Summary and future work; and, Wrap-up.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
FY20 ASC IC L2 Milestone 7180: Performance Portability of SIERRA Mechanics Applications to ATS-1 and ATS-2. Executive Summary
The overall goal of this work was to accelerate simulations supporting the nuclear deterrence (ND) mission through improved performance of key algorithms in the ASC IC Sierra multi-physics application suite. This work focused on porting and optimizing algorithms for the graphics processing units (GPU) on the second ASC advanced technology system (ATS-2), while maintaining or improving performance on commodity technology systems (CTS) and ATS-1. Furthermore, these algorithmic developments used the ASC developed Kokkos performance portability abstraction library to maintain high performance across platforms using identical code, and enable sustainable reduced-cost migration and performance optimization to emerging hardware.
97 MATHEMATICS AND COMPUTING↗
TChem v2.0 - A Software Toolkit for the Analysis of Complex Kinetic Models
TChem is an open source software library for solving complex computational chemistry problems and analyzing detailed chemical kinetic models. The software provides support for: complex kinetic models for gas-phase and surface chemistry; thermodynamic properties based on NASA polynomials; species production/consumption rates; stable time integrator for solving stiff time ordinary differential equations; and, reactor models such as homogenous gas-phase ignition (with analytical Jacobian matrices), continuously stirred tank reactor, plug-flow reactor. This toolkit builds upon earlier versions that were written in C and featured tools for gas-phase chemistry only. The current version of the software was completely refactored in C++, uses an object-oriented programming model, and adopts Kokkos as its portability layer to make it ready for the next generation computing architectures i.e., multi/many core computing platforms with GPU accelerators. We have expanded the range of kinetic models to include surface chemistry and have added examples pertaining to Continuously Stirred Tank Reactors (CSTR) and Plug Flow Reactor (PFR) models to complement the homogenous ignition examples present in the earlier versions. To exploit the massive parallelism available from modern computing platforms, the current software interface is designed to evaluate samples in parallel, which enables large scale parametric studies, e.g. for sensitivity analysis and model calibration.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Batched Sparse Linear Algebra (Final Report for Subcontract B648960)
This report finalizes design specifications for developing batched kernels for small tensor operations for unassembled matrix-free iterative solvers, batched solvers for partially assembled operators, and batched solvers with support for various sparse formats. The outcome of the project milestones is a set of interfaces to Batched Sparse LA solvers running on hardware accelerators for use in ECP Libraries and Applications. It is part of the development of sparse batched kernels, solvers/preconditioners as well as creating interoperability in xSDK libraries with sparse and dense batched functions to benefit ECP applications. The participants included representatives from ECP libraries (not limited to the xSDK project), applications, and vendors (AMD, Intel, and NVIDIA). Batched sparse linear algebra solvers form the new frontier for algorithmic development and performance engineering. Many applications (ECP and non-ECP alike) require simultaneous solutions of small linear systems of equations that are structurally sparse. To move towards high hardware utilization, it is important to provide these applications with appropriate interfaces to efficient batched sparse solvers running on modern hardware accelerators. We present interface designs in use by HPC software libraries supporting batched sparse linear algebra and the development of sparse batched kernel codes for solvers and preconditioners. We also address the potential interoperability opportunities to keep the software portable between the major hardware accelerators from AMD, Intel, and NVIDIA. The presented interface specifications includes batched band, sparse iterative, and sparse direct solvers. This report summarizes progress in Kokkos Kernels and the xSDK libraries MAGMA, Ginkgo, hypre, SUNDIALS, and SuperLU_dist.
97 MATHEMATICS AND COMPUTING↗
GPU Acceleration of a Diagnostic Wind Solver
Reducing QUIC-Fire simulation runtimes is crucial in enabling simulation ensembles to guide science-driven prescribed fire planning in regions of complex terrain. Utilizing GPUs through OpenACC and Kokkos frameworks to accelerate generation of 3D windfields in QUIC-Fire would provide substantial speedup to the runtime of the code. Using GPUs for the Successive Over-Relaxation (SOR) algorithm utilized in QUIC-Fire will require deriving a ‘four colored’ version of the well explored ‘Red-Black’ SOR parallelization scheme. This effort could open the door for utilizing GPUs in other portions of the QUIC-Fire algorithm, increasing its viability in more complex and larger domains.
58 GEOSCIENCES↗
Milestone 49 Report: Batched Sparse LA Phase 5 Implementation
Batched sparse linear algebra operations in general, and solvers in particular, have become the major algorithmic development activity and foremost performance engineering effort in the numerical software libraries work on modern hardware with accelerators such as GPUs. Many applications, ECP and non-ECP alike, require simultaneous solutions of many small linear systems of equations that are structurally sparse in one form or another. In order to move towards high hardware utilization levels, it is important to provide these applications with appropriate interface designs to be both functionally efficient and performance portable and give full access to the appropriate batched sparse solvers running on modern hardware accelerators prevalent across DOE supercomputing sites since the inception of ECP. To this end, we present here a summary of recent advances on the interface designs in use by HPC software libraries supporting batched sparse linear algebra and the development of sparse batched kernel codes for solvers and preconditioners. We also address the potential interoperability opportunities to keep the corresponding software portable between the major hardware accelerators from AMD, Intel, and NVIDIA, while maintaining the appropriate disclosure levels conforming to the active NDA agreements. The presented interface specifications include a mix of batched band, sparse iterative, and sparse direct solvers with their accompanying functionality that is already required by the application codes or we anticipated to be needed in the near future. This report summarizes progress in Kokkos Kernels and the xSDK libraries MAGMA, Ginkgo, hypre, PETSc, and SuperLU.
97 MATHEMATICS AND COMPUTING↗
Thermonuclear Burn in a Multiphysics Code on GPUs
Multiphysics codes links to a library called SINGE for the calculation of thermonuclear (TN) burn rates, but some current multiphysics codes do not attempt to leverage the support for parallel operation that SINGE provides. Our goal is to investigate implementations of the SINGE workflow and analyze how the use of a performance portability layer could reduce run time on CPU archi tectures while also supporting GPU architectures without requiring code modifications. We looked to the Kokkos C++ Performance Portability Ecosystem to implement hardware agnostic parallel patterns.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Trillion-Atom Exascale Performance Portability of FLARE for Catalysis
Presentation on grad school work scaling FLARE with LAMMPS and Kokkos to trillion atoms
Johansson, Anders [Sandia National Laboratories (S↗
Developments in Performance and Portability for MadGraph5_aMC@NLO
Event generators simulate particle interactions using Monte Carlo techniques, providing the primary connection between experiment and theory in experimental high energy physics. These software packages, which are the first step in the simulation worflow of collider experiments, represent approximately 5 to 20% of the annual WLCG usage for the ATLAS and CMS experiments. With computing architectures becoming more heterogeneous, it is important to ensure that these key software frameworks can be run on future systems, large and small. In this contribution, recent progress on porting and speeding up the Madgraph5_aMC@NLO event generator on hybrid architectures, i.e. CPU with GPU accelerators, is discussed. The main focus of this work has been in the calculation of scattering amplitudes and "matrix elements", which is the computational bottleneck of an event generation application. For physics processes limited to QCD leading order, the code generation toolkit has been expanded to produce matrix element calculations using C++ vector instructions on CPUs and using CUDA for NVidia GPUs, as well as using Alpaka, Kokkos and SYCL for multiple CPU and GPU architectures. Performance is reported in terms of matrix element calculations per time on NVidia, Intel, and AMD devices. The status and outlook for the integration of this work into a production release usable by the LHC experiments, with the same functionalities and very similar user interfaces as the current Fortran version, is also described.
Valassi, Andrea↗
A Scaling Study for Incompressible Multispecies Solver in Vertex-CFD
Multispecies incompressible flows occur widely in engineering and environmental applications, such as chemical reactors, fuel cells, ocean mixing, and biomedical systems. However, accurately resolving the complex transport and mixing phenomena associated with multiple interacting species remains computationally challenging, especially for large-scale problems. In this study, we present a robust, high-performance computing--enabled multispecies incompressible Navier–Stokes solver integrated within the Vertex-CFD framework. Our solver employs a fully coupled, implicit, finite element--based formulation that accurately captures the advection, diffusion, and interaction of multiple species in incompressible flows by leveraging the Kokkos library for parallel computing to achieve high computational efficiency. For pressure coupling, the entropically damped artificial compressibility method is utilized. We validated the solver against canonical test cases, including multispecies advection, diffusion, and Bateman systems; the results demonstrate second- and third-order spatial accuracy and consistent convergence. Additionally, we demonstrated the strong and weak scaling study results obtained on the leadership-class high-performance computing system, Frontier at Oak Ridge National Laboratory.
Oz, Furkan [ORNL] (ORCID:0000000265831724)↗
On a Simplified Approach to Achieve Parallel Performance and Portability Across CPU and GPU Architectures
This paper presents software advances to easily exploit computer architectures consisting of a multi-core CPU and CPU+GPU to accelerate diverse types of high-performance computing (HPC) applications using a single code implementation. The paper describes and demonstrates the performance of the open-source C++ matrix and array (MATAR) library that uniquely offers: (1) a straightforward syntax for programming productivity, (2) usable data structures for data-oriented programming (DOP) for performance, and (3) a simple interface to the open-source C++ Kokkos library for portability and memory management across CPUs and GPUs. The portability across architectures with a single code implementation is achieved by automatically switching between diverse fine-grained parallelism backends (e.g., CUDA, HIP, OpenMP, pthreads, etc.) at compile time. The MATAR library solves many longstanding challenges associated with easily writing software that can run in parallel on any computer architecture. This work benefits projects seeking to write new C++ codes while also addressing the challenges of quickly making existing Fortran codes performant and portable over modern computer architectures with minimal syntactical changes from Fortran to C++. We demonstrate the feasibility of readily writing new C++ codes and modernizing existing codes with MATAR to be performant, parallel, and portable across diverse computer architectures.
97 MATHEMATICS AND COMPUTING↗