Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “GPU Programming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

Final Report: High Energy Physics at the Energy Frontier at Louisiana Tech

The Louisiana Tech University High Energy Physics group has developed a research program aimed at experimentally testing the Standard Model of particle physics and searching for new phenomena through a focused set of analyses in collaboration with the ATLAS experiment at the Large Hadron Collider (LHC) at the CERN laboratory in Geneva. This research program includes involvement in the current operation and maintenance of the ATLAS experiment and full involvement in Phase 1 and Phase 2 upgrades in preparation for future high luminosity (HL-LHC) operation of the LHC. Our focus is solely on the ATLAS experiment at the LHC, with some related detector development and software efforts. We have established important service roles on ATLAS in five major areas: Triggers, especially jet triggers; Data Quality monitoring; grid computing; GPU applications for upgrades; and radiation testing for upgrades. Our physics research is focused on multijet measurements and top quark physics in final states containing tau leptons, which we propose to extend into related searches for new phenomena. Focusing on closely related topics in the jet and top analyses and coordinating these analyses in our group has led to high efficiency and increased visibility inside the ATLAS collaboration and beyond. Based on our work in the DØ experiment in Run II of the Fermilab Tevatron Collider, Louisiana Tech has developed a reputation as one of the leading institutions pursuing jet physics studies. Currently we are applying this expertise to the ATLAS experiment, with several multijet analyses in progress.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

The Kokkos OpenMPTarget Backend: Implementation and Lessons Learned

As the supercomputing landscape diversifies, solutions such as Kokkos to write vendor agnostic applications and libraries have risen in popularity. Kokkos provides a programming model designed for performance portability, which allows developers to write a single source implementation that can run efficiently on various architectures. At its heart, Kokkos maps parallel algorithms to architecture and vendor specific backends written in lower level programming models such as CUDA and HIP. Another approach to writing vendor agnostic parallel code is using OpenMP’s directives based approach, which lets developers annotate code to express parallelism. It is implemented at the compiler level and is supported by all major high performance computing vendors, as well as the primary Open Source toolchains GNU and LLVM. Since its inception, Kokkos has used OpenMP to parallelize on CPU architectures. In this paper, we explore leveraging OpenMP for a GPU backend and discuss the challenges we encountered when mapping the Kokkos APIs and semantics to OpenMP target constructs. As an exemplar workload we chose a simple conjugate gradient solver for sparse matrices. We find that performance on NVIDIA and AMD GPUs varies widely based on details of the implementation strategy and the chosen compiler. Furthermore, the performance of the OpenMP implementations decreases with increasing complexity of the investigated algorithms.

Gayatri, Rahulkumar↗

Benchmarking Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this paper, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, use of local memory, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗

AstraAI v1

AstraAI is an open-source, structure-aware AI coding agent designed for large scientific and DOE-HPC codebases such as AMReX-based applications. Unlike general-purpose coding assistants, AstraAI combines retrieval-augmented generation (RAG) with compiler-level Abstract Syntax Tree (AST) analysis to perform precise, scope-constrained code modifications. It identifies exact function spans, enforces locality of edits, and maintains cross-file invariants, enabling deterministic and build-safe transformations in complex C++/GPU environments. AstraAI is intended for developers working on large, evolving HPC frameworks where correctness, reproducibility, and structural integrity are critical. Typical use cases include modifying physics kernels, updating GPU device lambdas, and performing multi-file refactors without breaking compilation or runtime semantics. Compared to conventional LLM-based coding agents - even those with repository access - AstraAI provides structural guarantees rather than free-form text patches. It minimizes unintended diffs, prevents scope drift, preserves formatting and build stability, and reduces structural hallucinations. By integrating compiler tooling directly into the generation loop, AstraAI transforms AI-assisted coding from probabilistic text editing into deterministic, structure-preserving program transformation suitable for mission-critical scientific software.

Natarajan, Mahesh [Lawrence Berkeley National Labo↗

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this work, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, shared local memory accesses, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

97 MATHEMATICS AND COMPUTING↗

Dissecting Tensor Cores via Microbenchmarks: Latency, Throughput and Numeric Behaviors

Tensor Cores have been an important unit to accelerate Fused Matrix Multiplication Accumulation (MMA) in all NVIDIA GPUs since Volta Architecture. To program Tensor Cores, users have to use either legacy wmma APIs or current mma APIs. Legacy wmma APIs are more easy-to-use but can only exploit limited features and power of Tensor Cores. Specifically, wmma APIs support fewer operand shapes and can not leverage the new sparse matrix multiplication feature of the newest Ampere Tensor Cores. However, the performance of current programming interface has not been well explored. Furthermore, the computation numeric behaviors of low-precision floating points (TF32, BF16, and FP16) supported by the newest Ampere Tensor Cores are also mysterious. In this paper, we explore the throughput and latency of current programming APIs. Further, we intuitively study the numeric behaviors of Tensor Cores MMA and profile the intermediate operations including multiplication, addition of inner product, and accumulation. All codes used in this work can be found in https://github.com/sunlex0717/DissectingTensorCores.

97 MATHEMATICS AND COMPUTING↗

The Importance of Scientific Visualization on Novel Hardware

Innovation in HPC hardware and adoption of heterogeneous systems has led to a variety of unique programming models. This has led to a challenge for scientific visualization software (and indeed all HPC software) to take full advantage of recent generations of supercomputing. Edge computing, where hardware is specialized for the needs of the particular application, exacerbates the problem. VTK-m has had many successes on this front by providing device-agnostic algorithms that compare favorably to implementations written to specific devices as shown in Table 1. However, VTK-m has focused mostly on GPU and traditional CPU multicore technology. There are numerous processor technologies, both existing and potential future, that are not being addressed by current R&D efforts.

97 MATHEMATICS AND COMPUTING↗

Roofline model toolkit: A practical tool for architectural and program analysis

We present preliminary results of the Roofline Toolkit for multicore, many core, and accelerated architectures. This paper focuses on the processor architecture characterization engine, a collection of portable instrumented micro benchmarks implemented with Message Passing Interface (MPI), and OpenMP used to express thread-level parallelism. These benchmarks are specialized to quantify the behavior of different architectural features. Compared to previous work on performance characterization, these microbenchmarks focus on capturing the performance of each level of the memory hierarchy, along with thread-level parallelism, instruction-level parallelism and explicit SIMD parallelism, measured in the context of the compilers and run-time environments. We also measure sustained PCIe throughput with four GPU memory managed mechanisms. By combining results from the architecture characterization with the Roofline model based solely on architectural specifications, this work offers insights for performance prediction of current and future architectures and their software systems. To that end, we instrument three applications and plot their resultant performance on the corresponding Roofline model when run on a Blue Gene/Q architecture.

97 MATHEMATICS AND COMPUTING↗

TU-C-HORIZONS-01: The Expanding Horizons Travel Grant Program: ePosters and Discussion

The Expanding Horizons travel grant program provides opportunity for students and trainees to broaden the scope of scientific meetings they attend and gain insight from research outside traditional domains of medical physics. Through participation in such conferences, early-career researchers are introduced to new topics with relevance to medical physics research as a means to expand the scientific horizons of our discipline. This year, 21 Expanding Horizons travel grants were awarded, granting travel to 17 conferences, including: Radiomics, the World Molecular Imaging Society (WMIS), the 3D Printing Conference and Expo, the GPU Technology Conference, the SIAM Imaging Science Conference, the Human Brain Mapping Conference, the OSA Conference on Clinical and Translational Biophotonics, the Society for Neuroscience, the AACR Conference on Tumor Microenvironment, and the Conference on Knowledge Discovery and Data Mining. The Expanding Horizons electronic poster session gives a venue for AAPM conference attendees to meet and discuss with awardees, learn the hot topics and emerging research areas presented at these conferences, and understand the relevance to future medical physics research.

61 RADIATION PROTECTION AND DOSIMETRY↗

Software Quality Assurance for the MOOSE-Based Open-Source Multiphysics Code Cardinal - An Expanded CI Testing Suite

Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing and, maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program has been initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as a safety analysis tool. These gaps have been captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents a snapshot of the verification test cases currently available for Cardinal and NekRS in their assimilation into a Continuous Integration (CI) platform. Following the CI practice permits the integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. It should be noted that the SQA program itself, including the program plans, procedures, configuration management, and testing strategies, need to be developed in a future step of this task.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Progress Towards NQA-1 for Cardinal in FY25

Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing,and maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program was initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as an analysis tool. These gaps were captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents the progress made towards NQA-1 for Cardinal in the Fiscal Year 2025 (FY25). All cases in the expanded Continuous Integration (CI) suite of NekRS are included in this report which test the solvers and modules available in NekRS exhaustively. The NekRS tests are integrated with the Cardinal CI suite and made available in publicly accessible Github documentation. Following the CI practice permits integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. Also in this report is a brief overview of the development of the Cardinal Software Quality Assurance Plan (SQAP) that was done in FY25, though it should be noted that the rest of the documentation for the SQA program needs to be developed in a future step of this task.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Extreme-Scale Stochastic Particle Tracing for Uncertain Unsteady Flow Visualization and Analysis

We present an efficient and scalable solution to estimate uncertain transport behaviors-stochastic flow maps (SFMs)-for visualizing and analyzing uncertain unsteady flows. Computing flow maps from uncertain flow fields is extremely expensive because it requires many Monte Carlo runs to trace densely seeded particles in the flow. We reduce the computational cost by decoupling the time dependencies in SFMs so that we can process shorter sub time intervals independently and then compose them together for longer time periods. Adaptive refinement is also used to reduce the number of runs for each location. We parallelize over tasks-packets of particles in our design-to achieve high efficiency in MPI/thread hybrid programming. Such a task model also enables CPU/GPU coprocessing. We show the scalability on two supercomputers, Mira (up to 256K Blue Gene/Q cores) and Titan (up to 128K Opteron cores and 8K GPUs), that can trace billions of particles in seconds.

CPU-GPU hybrid parallelism↗

kynema-fmb [SWR-23-07]

Kynema-FMB (FKA: Kynema) is an open-source performance portable flexible multibody (FMB) dynamics solver designed for time-domain simulations. While originally tailored for wind turbine structural dynamics, the formulation and implementation are those of a general flexible-multidbody dynamics solver that can readily be applied to a wide range of systems. Kynema was designed with a narrow focus, namely to provide a lightweight, fast, accurate FMD solver for coupling to computational-fluid-dynamics (CFD) codes, especially the CFD codes in the Kynema suite, for fluid-structure-interaction (FSI) simulations. Kynema-FMB is equipped to model systems that can be represented as a collection of beams and rigid bodies that are connected through constraints. Degrees of freedom are defined in the inertial/global frame of reference and include displacements and rotations (formally as rotation matrices, but stored as quaternions). The underlying formulation is built on a Lie-group time integrator designed for index-3 differential-algebraic equations, which is second-order accurate in time (Bruls et al., 2012). Beam models are based on geometrically exact beam theory and are discretized as high-order spectral finite elements similar to those in BeamDyn (Wang et al., 2017). The governing equations for a FMD system like a wind turbine constitute a highly nonlinear system of constrained partial-differential equations. Kynema-FMB uses analytical Jacobians in the nonlinear-system solves in each time step. Linear systems use sparse storage and several third-party sparse-linear-system solvers are enabled. Ill conditioning of linear systems is mitigated with preconditioning described in Bottasso et al, 2008. Kynema-FMB is integrated with a simple open-source controller (ROSCO). There is an application programming interface (API) for coupling to geometry-resolved CFD (like that in Sharma et al., 2023) and actuator-force CFD (like that in Kuhn et al., 2025). In the latter, for actuator-line models, Kynema-FMB includes an internal blade-element solver that depends on user-provided lookup tables for coefficients of lift and drag, i.e., aerodynamic polars. Kynema-FMB is written in C++ and leverages Kokkos and Kokkos-Kernels (KokkosEcosystem) as its performance portability layer enabling simulations on both CPU and GPU systems. The repository is equipped with extensive automated testing at the unit and regression/system levels. The following describes the high-level development objectives conceived for Kynema: *Kynema will follow modern software development best practices, including test-driven development (TDD), version control, hierarchical automated testing, and continuous integration (CI) for a robust development environment. *The core data structures are memory efficient and enable vectorization and parallelization at multiple levels. *Data structures are data-oriented to exploit methods for accelerated computing including high utilization of chip resources (e.g., single instruction multiple data (SIMD) instruction sets) and parallelization using GP-GPUs. *The computational algorithms incorporate robust open-source libraries for mathematical operations, resource allocation, and data management. *The API design considers multiple stakeholder needs and ensure integration with existing and future ecosystems for data science, machine learning, and AI. *Kynema-FMB is written in modern C++ and leverages Kokkos as its performance-portability library with inspiration from the kynema stack.

Sprague, MichaelA.↗

Assimilation of Wave Imaging Radar Observations for Real-time Wave-by-Wave Forecasting

This project addressed Topic 3: “Wave Measurement Instrumentation for Feed Forward Controls” under the FOA number DE-FOA-0000971. The overall goal of the program was to develop a phase-resolving wave forecasting technique for application to the active control of Wave Energy Conversion (WEC) devices. We have developed an approach that couples a wave imaging marine radar with a phase-resolving linear wave model for real-time wave field reconstruction and forward propagation of the wave field in space and time. The scope of the project was to develop and assess the performance of this novel forecasting system. Specific project goals were as follows: Develop and verify a fast, GPU-based (Graphical Processing Unit) wave propagation model suitable for phase-resolved computation of nearshore wave transformation over variable bathymetry; Compare the accuracy and speed of performance of the wave model against a deep water model in their ability to predict wave field transformation in the intermediate water depths (50 to 70 m) typical of planned WEC sites; Develop and implement a variational assimilation algorithm that can ingest wave imaging radar observations and estimate the time-varying wave conditions offshore of the domain of interest such that the observed wave field is best reconstructed throughout the domain and then use this to produce model forecasts for a given WEC location; Collect wave-resolving marine radar data, along with relevant in situ wave data, at a suitable wave energy test site, apply the algorithm to the field data, assess performance, and identify any necessary improvements; and Develop a production cost estimate that addresses the affordability of the wave forecasting technology and include in the Final Report. The developed forecasting algorithm (“Wavecast”) was evaluated for both speed and accuracy against a substantial synthetic dataset. Early in the project, performance tests definitively demonstrated that the system was capable of forecasting in real-time, as the GPU-based wave model backbone was very computationally efficient. The data assimilation algorithm was developed on a polar grid domain in order to match the sampling characteristics of the observation system (wave imaging marine radar). For verification purposes, a substantial set of synthetic wave data (i.e. forward runs of the wave model) were generated to be used as ground truth for comparison to the reconstructions and forecasts produced by Wavecast. For these synthetic cases, Wavecast demonstrated very good accuracy, for example, typical forecast correlation coefficients were between 0.84-0.95 when compared to the input data. Dependencies on shadowing, observational noise, and forecast horizon were also identified. During the second year of the project, a short field deployment was conducted in order to assess forecast accuracy under field conditions. For this, a radar was installed on a fishing vessel and observations were collected at the South Energy Test Site (SETS) off the coast of Newport, OR. At the SETS site, simultaneous in situ wave observations were also available owing to an ongoing field project funded separately. Unfortunately, the position and heading information that was available for the fishing vessel were not of sufficient accuracy in order to validate the forecast in a phase-resolving sense. Instead, a spectral comparison was made between the Wavecast forecast and the data from the in situ wave buoy. Although the wave and wind conditions during the field test were complex, the comparison showed a promising reconstruction of the wave spectral shape, where both peaks in the bimodal spectrum were represented. However, the total reconstructed spectral energy (across all directions and frequencies) was limited to 44% of the observed spectrum. Overall, wave-by-wave forecasting using a data assimilation approach based on wave imaging radar observations and a physics-based wave model shows promise for short-term phase-resolved predictions. Two recommendations for future work are as follows: first, we would recommend additional focused field campaigns for algorithm validation. The field campaign should be long enough to capture a range of wave conditions relevant to the target application and WEC site. In addition, it will be crucial to make sure the vessel of choice has high accuracy position and heading instrumentation (this instrumentation is commercially available but not standard on commercial fishing vessels). The second recommendation is to expand the model physics in the wave model backbone to include some nonlinear effects. Specifically, the third-order correction to the wave speed due to amplitude dispersion would be the next step in order to more accurately represent the phase speeds of large amplitude waves.

16 TIDAL AND WAVE POWER↗

Comparing the Performance of Julia on CPUs versus GPUs and Julia-MPI versus Fortran-MPI: a case study with MPAS-Ocean (Version 7.1)

Abstract. Some programming languages are easy to develop at the cost of slow execution, while others are fast at runtime but much more difficult to write. Julia is a programming language that aims to be the best of both worlds – a development and production language at the same time. To test Julia's utility in scientific high-performance computing (HPC), we built an unstructured-mesh shallow water model in Julia and compared it against an established Fortran-MPI ocean model, the Model for Prediction Across Scales–Ocean (MPAS-Ocean), as well as a Python shallow water code. Three versions of the Julia shallow water code were created: for single-core CPU, graphics processing unit (GPU), and Message Passing Interface (MPI) CPU clusters. Comparing identical simulations revealed that our first version of the Julia model was 13 times faster than Python using NumPy, where both used an unthreaded single-core CPU. Further Julia optimizations, including static typing and removing implicit memory allocations, provided an additional 10–20× speed-up of the single-core CPU Julia model. The GPU-accelerated Julia code was almost identical in terms of performance to the MPI parallelized code on 64 processes, an unexpected result for such different architectures. Parallelized Julia-MPI performance was identical to Fortran-MPI MPAS-Ocean for low processor counts and ranges from 2× faster to 2× slower for higher processor counts. Our experience is that Julia development is fast and convenient for prototyping but that Julia requires further investment and expertise to be competitive with compiled codes. We provide advice on Julia code optimization for HPC systems.

54 ENVIRONMENTAL SCIENCES↗

Abstractions and Directives for Adapting Wavefront Algorithms to Future Architectures

Architectures are rapidly evolving, and exascale machines are expected to offer billion-way concurrency. We need to rethink algorithms, languages and programming models among other components in order to migrate large scale applications and explore parallelism on these machines. Although directive-based programming models allow programmers to worry less about programming and more about science, expressing complex parallel patterns in these models can be a daunting task especially when the goal is to match the performance that the hardware platforms can offer. One such pattern is wavefront. This paper extensively studies a wavefront-based miniapplication for Denovo, a production code for nuclear reactor modeling. We parallelize the Koch-Baker-Alcouffe (KBA) parallel-wavefront sweep algorithm in the main kernel of Minisweep (the miniapplication) using CUDA, OpenMP and OpenACC. Our OpenACC implementation running on NVIDIA's next-generation Volta GPU boasts an 85.06x speedup over serial code, which is larger than CUDA's 83.72x speedup over the same serial implementation. Our experimental platform includes SummitDev, an ORNL representative architecture of the upcoming Summit supercomputer. Our parallelization effort across platforms also motivated us to define an abstract parallelism model that is architecture independent, with a goal of creating software abstractions that can be used by applications employing the wavefront sweep motif.

Searles, Robert↗

MPI + OpenACC: Accelerating radiation transport mini-application, minisweep, on heterogeneous systems

According to the article, architectures are rapidly evolving, and exascale machines are expected to offer billion-way concurrency. We need to rethink algorithms, languages and programming models among other components in order to migrate large scale applications and explore parallelism on these machines. Although directive-based programming models allow programmers to worry less about programming and more about science, expressing complex parallel patterns in these models can be a daunting task especially when the goal is to match the performance that the hardware platforms can offer. One such pattern is wavefront. This paper extensively studies a wavefront-based miniapplication for Denovo, a production code for nuclear reactor modeling. We parallelize the Koch–Baker–Alcouffe (KBA) parallel-wavefront sweep algorithm in the main kernel of Minisweep (the miniapplication) using CUDA 9.0, OpenMP 4.0 (SIMD) and OpenACC 2.6. Our OpenACC implementation running on NVIDIA’s next-generation Volta GPU boasts an 85.06x speedup over serial code, which is larger than CUDA’s 83.72x speedup over the same serial implementation. We also explore the scalability of our solution using MPI to decompose our simulation domain, allowing us to run on many nodes and accelerators present in state-of-the-art HPC systems. Our parallelization effort across platforms also motivated us to define an abstract parallelism model that is architecture independent, with a goal of creating software abstractions that can be used by applications employing the wavefront sweep motif.

Computer Science↗

A feasibility study on porting the community land model onto accelerators using OpenACC

As environmental models (such as Accelerated Climate Model for Energy (ACME), Parallel Reactive Flow and Transport Model (PFLOTRAN), Arctic Terrestrial Simulator (ATS), etc.) became more and more complicated, we are facing enormous challenges regarding to porting those applications onto hybrid computing architecture. OpenACC appears as a very promising technology, therefore, we have conducted a feasibility analysis on porting the Community Land Model (CLM), a terrestrial ecosystem model within the Community Earth System Models (CESM)). Specifically, we used automatic function testing platform to extract a small computing kernel out of CLM, then we apply this kernel into the actually CLM dataflow procedure, and investigate the strategy of data parallelization and the benefit of data movement provided by current implementation of OpenACC. Even it is a non-intensive kernel, on a single 16-core computing node, the performance (based on the actual computation time using one GPU) of OpenACC implementation is 2.3 time faster than that of OpenMP implementation using single OpenMP thread, but it is 2.8 times slower than the performance of OpenMP implementation using 16 threads. On multiple nodes, MPI_OpenACC implementation demonstrated very good scalability on up to 128 GPUs on 128 computing nodes. This study also provides useful information for us to look into the potential benefits of “deep copy” capability and “routine” feature of OpenACC standards. In conclusion, we believe that our experience on the environmental model, CLM, can be beneficial to many other scientific research programs who are interested to porting their large scale scientific code using OpenACC onto high-end computers, empowered by hybrid computing architecture.

54 ENVIRONMENTAL SCIENCES↗