Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “GPU Programming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Dissecting Tensor Cores via Microbenchmarks: Latency, Throughput and Numeric Behaviors

Tensor Cores have been an important unit to accelerate Fused Matrix Multiplication Accumulation (MMA) in all NVIDIA GPUs since Volta Architecture. To program Tensor Cores, users have to use either legacy wmma APIs or current mma APIs. Legacy wmma APIs are more easy-to-use but can only exploit limited features and power of Tensor Cores. Specifically, wmma APIs support fewer operand shapes and can not leverage the new sparse matrix multiplication feature of the newest Ampere Tensor Cores. However, the performance of current programming interface has not been well explored. Furthermore, the computation numeric behaviors of low-precision floating points (TF32, BF16, and FP16) supported by the newest Ampere Tensor Cores are also mysterious. In this paper, we explore the throughput and latency of current programming APIs. Further, we intuitively study the numeric behaviors of Tensor Cores MMA and profile the intermediate operations including multiplication, addition of inner product, and accumulation. All codes used in this work can be found in https://github.com/sunlex0717/DissectingTensorCores.

97 MATHEMATICS AND COMPUTING↗

The Importance of Scientific Visualization on Novel Hardware

Innovation in HPC hardware and adoption of heterogeneous systems has led to a variety of unique programming models. This has led to a challenge for scientific visualization software (and indeed all HPC software) to take full advantage of recent generations of supercomputing. Edge computing, where hardware is specialized for the needs of the particular application, exacerbates the problem. VTK-m has had many successes on this front by providing device-agnostic algorithms that compare favorably to implementations written to specific devices as shown in Table 1. However, VTK-m has focused mostly on GPU and traditional CPU multicore technology. There are numerous processor technologies, both existing and potential future, that are not being addressed by current R&D efforts.

97 MATHEMATICS AND COMPUTING↗

Roofline model toolkit: A practical tool for architectural and program analysis

We present preliminary results of the Roofline Toolkit for multicore, many core, and accelerated architectures. This paper focuses on the processor architecture characterization engine, a collection of portable instrumented micro benchmarks implemented with Message Passing Interface (MPI), and OpenMP used to express thread-level parallelism. These benchmarks are specialized to quantify the behavior of different architectural features. Compared to previous work on performance characterization, these microbenchmarks focus on capturing the performance of each level of the memory hierarchy, along with thread-level parallelism, instruction-level parallelism and explicit SIMD parallelism, measured in the context of the compilers and run-time environments. We also measure sustained PCIe throughput with four GPU memory managed mechanisms. By combining results from the architecture characterization with the Roofline model based solely on architectural specifications, this work offers insights for performance prediction of current and future architectures and their software systems. To that end, we instrument three applications and plot their resultant performance on the corresponding Roofline model when run on a Blue Gene/Q architecture.

97 MATHEMATICS AND COMPUTING↗

TU-C-HORIZONS-01: The Expanding Horizons Travel Grant Program: ePosters and Discussion

The Expanding Horizons travel grant program provides opportunity for students and trainees to broaden the scope of scientific meetings they attend and gain insight from research outside traditional domains of medical physics. Through participation in such conferences, early-career researchers are introduced to new topics with relevance to medical physics research as a means to expand the scientific horizons of our discipline. This year, 21 Expanding Horizons travel grants were awarded, granting travel to 17 conferences, including: Radiomics, the World Molecular Imaging Society (WMIS), the 3D Printing Conference and Expo, the GPU Technology Conference, the SIAM Imaging Science Conference, the Human Brain Mapping Conference, the OSA Conference on Clinical and Translational Biophotonics, the Society for Neuroscience, the AACR Conference on Tumor Microenvironment, and the Conference on Knowledge Discovery and Data Mining. The Expanding Horizons electronic poster session gives a venue for AAPM conference attendees to meet and discuss with awardees, learn the hot topics and emerging research areas presented at these conferences, and understand the relevance to future medical physics research.

61 RADIATION PROTECTION AND DOSIMETRY↗

Celestial Navigation in Cislunar Space with autoNGC

Celestial navigation (CelNav) is a source of navigation observables where images of known solar system bodies are used to locate a spacecraft, beneficial within the solar system for both cislunar and deep space missions. CelNav provides a variety of design benefits to support and enable current and new autonomous space operations- using only a camera and a processor to produce in-situ measurements for navigation. This technology reduces subscription to ground-based tracking during all phases of a mission, freeing up resources for other operational needs. This also supports secure navigation since it eliminates the need for ground contact. CelNav enables missions where the light time delay between Earth and the spacecraft is too long (or the Earth to spacecraft line of sight is obscured) to support critical operations. It also enables smaller mission classes, where Deep Space Network (DSN)time is cost prohibitive, to reduce its cost by focusing primarily on data downlink. Finally, it enables the NASA Artemis program and other cislunar human space flight by providing redundant navigation to traditional radiometric tracking. In this presentation, we discuss the implementation of a CelNav app in autonomous Navigation, Guidance, and Control (autoNGC), a comprehensive flight software suite for onboard autonomy that is built on the core Flight System (cFS). The presentation also summarizes the results of flight software-in-the-loop (SIL) and processor-in-the-loop (PIL) demonstrations. Both are high-fidelity simulations with the use of a camera emulator hosted on a GPU server that simulates images that would be captured by the camera. The CelNav app leverages the use of cGIANT (cFS Goddard Image Analysis and Navigation Tool).Previously developed for the autoNGC software suite, cGIANT is an onboard autonomous image processing and optical navigation (OpNav) tool that performs limb-based OpNav and Terrain Relative Navigation. The added CelNav capability of cGIANT generates bearing measurements to multiple known celestial bodies (planets, moons, asteroids, comets, etc.) in monocular (2D) images. These observables are then fed to the Goddard Enhanced Onboard Navigation System (GEONS)navigation filter app, enabling us to navigate the spacecraft autonomously. In early 2025, the autoNGC CelNav capability is planned to be flight tested as part of the onboard autonomy experiment on the Cislunar Autonomous Positioning System Technology Operations and Navigation Experiment(CAPSTONE) spacecraft that is currently in a Lunar Near Rectilinear Halo Orbit(NRHO).

celestial navigation↗

Software Quality Assurance for the MOOSE-Based Open-Source Multiphysics Code Cardinal - An Expanded CI Testing Suite

Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing and, maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program has been initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as a safety analysis tool. These gaps have been captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents a snapshot of the verification test cases currently available for Cardinal and NekRS in their assimilation into a Continuous Integration (CI) platform. Following the CI practice permits the integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. It should be noted that the SQA program itself, including the program plans, procedures, configuration management, and testing strategies, need to be developed in a future step of this task.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Progress Towards NQA-1 for Cardinal in FY25

Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing,and maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program was initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as an analysis tool. These gaps were captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents the progress made towards NQA-1 for Cardinal in the Fiscal Year 2025 (FY25). All cases in the expanded Continuous Integration (CI) suite of NekRS are included in this report which test the solvers and modules available in NekRS exhaustively. The NekRS tests are integrated with the Cardinal CI suite and made available in publicly accessible Github documentation. Following the CI practice permits integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. Also in this report is a brief overview of the development of the Cardinal Software Quality Assurance Plan (SQAP) that was done in FY25, though it should be noted that the rest of the documentation for the SQA program needs to be developed in a future step of this task.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Extreme-Scale Stochastic Particle Tracing for Uncertain Unsteady Flow Visualization and Analysis

We present an efficient and scalable solution to estimate uncertain transport behaviors-stochastic flow maps (SFMs)-for visualizing and analyzing uncertain unsteady flows. Computing flow maps from uncertain flow fields is extremely expensive because it requires many Monte Carlo runs to trace densely seeded particles in the flow. We reduce the computational cost by decoupling the time dependencies in SFMs so that we can process shorter sub time intervals independently and then compose them together for longer time periods. Adaptive refinement is also used to reduce the number of runs for each location. We parallelize over tasks-packets of particles in our design-to achieve high efficiency in MPI/thread hybrid programming. Such a task model also enables CPU/GPU coprocessing. We show the scalability on two supercomputers, Mira (up to 256K Blue Gene/Q cores) and Titan (up to 128K Opteron cores and 8K GPUs), that can trace billions of particles in seconds.

CPU-GPU hybrid parallelism↗

kynema-fmb [SWR-23-07]

Kynema-FMB (FKA: Kynema) is an open-source performance portable flexible multibody (FMB) dynamics solver designed for time-domain simulations. While originally tailored for wind turbine structural dynamics, the formulation and implementation are those of a general flexible-multidbody dynamics solver that can readily be applied to a wide range of systems. Kynema was designed with a narrow focus, namely to provide a lightweight, fast, accurate FMD solver for coupling to computational-fluid-dynamics (CFD) codes, especially the CFD codes in the Kynema suite, for fluid-structure-interaction (FSI) simulations. Kynema-FMB is equipped to model systems that can be represented as a collection of beams and rigid bodies that are connected through constraints. Degrees of freedom are defined in the inertial/global frame of reference and include displacements and rotations (formally as rotation matrices, but stored as quaternions). The underlying formulation is built on a Lie-group time integrator designed for index-3 differential-algebraic equations, which is second-order accurate in time (Bruls et al., 2012). Beam models are based on geometrically exact beam theory and are discretized as high-order spectral finite elements similar to those in BeamDyn (Wang et al., 2017). The governing equations for a FMD system like a wind turbine constitute a highly nonlinear system of constrained partial-differential equations. Kynema-FMB uses analytical Jacobians in the nonlinear-system solves in each time step. Linear systems use sparse storage and several third-party sparse-linear-system solvers are enabled. Ill conditioning of linear systems is mitigated with preconditioning described in Bottasso et al, 2008. Kynema-FMB is integrated with a simple open-source controller (ROSCO). There is an application programming interface (API) for coupling to geometry-resolved CFD (like that in Sharma et al., 2023) and actuator-force CFD (like that in Kuhn et al., 2025). In the latter, for actuator-line models, Kynema-FMB includes an internal blade-element solver that depends on user-provided lookup tables for coefficients of lift and drag, i.e., aerodynamic polars. Kynema-FMB is written in C++ and leverages Kokkos and Kokkos-Kernels (KokkosEcosystem) as its performance portability layer enabling simulations on both CPU and GPU systems. The repository is equipped with extensive automated testing at the unit and regression/system levels. The following describes the high-level development objectives conceived for Kynema: *Kynema will follow modern software development best practices, including test-driven development (TDD), version control, hierarchical automated testing, and continuous integration (CI) for a robust development environment. *The core data structures are memory efficient and enable vectorization and parallelization at multiple levels. *Data structures are data-oriented to exploit methods for accelerated computing including high utilization of chip resources (e.g., single instruction multiple data (SIMD) instruction sets) and parallelization using GP-GPUs. *The computational algorithms incorporate robust open-source libraries for mathematical operations, resource allocation, and data management. *The API design considers multiple stakeholder needs and ensure integration with existing and future ecosystems for data science, machine learning, and AI. *Kynema-FMB is written in modern C++ and leverages Kokkos as its performance-portability library with inspiration from the kynema stack.

Sprague, MichaelA.↗

Assimilation of Wave Imaging Radar Observations for Real-time Wave-by-Wave Forecasting

This project addressed Topic 3: “Wave Measurement Instrumentation for Feed Forward Controls” under the FOA number DE-FOA-0000971. The overall goal of the program was to develop a phase-resolving wave forecasting technique for application to the active control of Wave Energy Conversion (WEC) devices. We have developed an approach that couples a wave imaging marine radar with a phase-resolving linear wave model for real-time wave field reconstruction and forward propagation of the wave field in space and time. The scope of the project was to develop and assess the performance of this novel forecasting system. Specific project goals were as follows: Develop and verify a fast, GPU-based (Graphical Processing Unit) wave propagation model suitable for phase-resolved computation of nearshore wave transformation over variable bathymetry; Compare the accuracy and speed of performance of the wave model against a deep water model in their ability to predict wave field transformation in the intermediate water depths (50 to 70 m) typical of planned WEC sites; Develop and implement a variational assimilation algorithm that can ingest wave imaging radar observations and estimate the time-varying wave conditions offshore of the domain of interest such that the observed wave field is best reconstructed throughout the domain and then use this to produce model forecasts for a given WEC location; Collect wave-resolving marine radar data, along with relevant in situ wave data, at a suitable wave energy test site, apply the algorithm to the field data, assess performance, and identify any necessary improvements; and Develop a production cost estimate that addresses the affordability of the wave forecasting technology and include in the Final Report. The developed forecasting algorithm (“Wavecast”) was evaluated for both speed and accuracy against a substantial synthetic dataset. Early in the project, performance tests definitively demonstrated that the system was capable of forecasting in real-time, as the GPU-based wave model backbone was very computationally efficient. The data assimilation algorithm was developed on a polar grid domain in order to match the sampling characteristics of the observation system (wave imaging marine radar). For verification purposes, a substantial set of synthetic wave data (i.e. forward runs of the wave model) were generated to be used as ground truth for comparison to the reconstructions and forecasts produced by Wavecast. For these synthetic cases, Wavecast demonstrated very good accuracy, for example, typical forecast correlation coefficients were between 0.84-0.95 when compared to the input data. Dependencies on shadowing, observational noise, and forecast horizon were also identified. During the second year of the project, a short field deployment was conducted in order to assess forecast accuracy under field conditions. For this, a radar was installed on a fishing vessel and observations were collected at the South Energy Test Site (SETS) off the coast of Newport, OR. At the SETS site, simultaneous in situ wave observations were also available owing to an ongoing field project funded separately. Unfortunately, the position and heading information that was available for the fishing vessel were not of sufficient accuracy in order to validate the forecast in a phase-resolving sense. Instead, a spectral comparison was made between the Wavecast forecast and the data from the in situ wave buoy. Although the wave and wind conditions during the field test were complex, the comparison showed a promising reconstruction of the wave spectral shape, where both peaks in the bimodal spectrum were represented. However, the total reconstructed spectral energy (across all directions and frequencies) was limited to 44% of the observed spectrum. Overall, wave-by-wave forecasting using a data assimilation approach based on wave imaging radar observations and a physics-based wave model shows promise for short-term phase-resolved predictions. Two recommendations for future work are as follows: first, we would recommend additional focused field campaigns for algorithm validation. The field campaign should be long enough to capture a range of wave conditions relevant to the target application and WEC site. In addition, it will be crucial to make sure the vessel of choice has high accuracy position and heading instrumentation (this instrumentation is commercially available but not standard on commercial fishing vessels). The second recommendation is to expand the model physics in the wave model backbone to include some nonlinear effects. Specifically, the third-order correction to the wave speed due to amplitude dispersion would be the next step in order to more accurately represent the phase speeds of large amplitude waves.

16 TIDAL AND WAVE POWER↗

Comparing the Performance of Julia on CPUs versus GPUs and Julia-MPI versus Fortran-MPI: a case study with MPAS-Ocean (Version 7.1)

Abstract. Some programming languages are easy to develop at the cost of slow execution, while others are fast at runtime but much more difficult to write. Julia is a programming language that aims to be the best of both worlds – a development and production language at the same time. To test Julia's utility in scientific high-performance computing (HPC), we built an unstructured-mesh shallow water model in Julia and compared it against an established Fortran-MPI ocean model, the Model for Prediction Across Scales–Ocean (MPAS-Ocean), as well as a Python shallow water code. Three versions of the Julia shallow water code were created: for single-core CPU, graphics processing unit (GPU), and Message Passing Interface (MPI) CPU clusters. Comparing identical simulations revealed that our first version of the Julia model was 13 times faster than Python using NumPy, where both used an unthreaded single-core CPU. Further Julia optimizations, including static typing and removing implicit memory allocations, provided an additional 10–20× speed-up of the single-core CPU Julia model. The GPU-accelerated Julia code was almost identical in terms of performance to the MPI parallelized code on 64 processes, an unexpected result for such different architectures. Parallelized Julia-MPI performance was identical to Fortran-MPI MPAS-Ocean for low processor counts and ranges from 2× faster to 2× slower for higher processor counts. Our experience is that Julia development is fast and convenient for prototyping but that Julia requires further investment and expertise to be competitive with compiled codes. We provide advice on Julia code optimization for HPC systems.

54 ENVIRONMENTAL SCIENCES↗

Abstractions and Directives for Adapting Wavefront Algorithms to Future Architectures

Architectures are rapidly evolving, and exascale machines are expected to offer billion-way concurrency. We need to rethink algorithms, languages and programming models among other components in order to migrate large scale applications and explore parallelism on these machines. Although directive-based programming models allow programmers to worry less about programming and more about science, expressing complex parallel patterns in these models can be a daunting task especially when the goal is to match the performance that the hardware platforms can offer. One such pattern is wavefront. This paper extensively studies a wavefront-based miniapplication for Denovo, a production code for nuclear reactor modeling. We parallelize the Koch-Baker-Alcouffe (KBA) parallel-wavefront sweep algorithm in the main kernel of Minisweep (the miniapplication) using CUDA, OpenMP and OpenACC. Our OpenACC implementation running on NVIDIA's next-generation Volta GPU boasts an 85.06x speedup over serial code, which is larger than CUDA's 83.72x speedup over the same serial implementation. Our experimental platform includes SummitDev, an ORNL representative architecture of the upcoming Summit supercomputer. Our parallelization effort across platforms also motivated us to define an abstract parallelism model that is architecture independent, with a goal of creating software abstractions that can be used by applications employing the wavefront sweep motif.

Searles, Robert↗

MPI + OpenACC: Accelerating radiation transport mini-application, minisweep, on heterogeneous systems

According to the article, architectures are rapidly evolving, and exascale machines are expected to offer billion-way concurrency. We need to rethink algorithms, languages and programming models among other components in order to migrate large scale applications and explore parallelism on these machines. Although directive-based programming models allow programmers to worry less about programming and more about science, expressing complex parallel patterns in these models can be a daunting task especially when the goal is to match the performance that the hardware platforms can offer. One such pattern is wavefront. This paper extensively studies a wavefront-based miniapplication for Denovo, a production code for nuclear reactor modeling. We parallelize the Koch–Baker–Alcouffe (KBA) parallel-wavefront sweep algorithm in the main kernel of Minisweep (the miniapplication) using CUDA 9.0, OpenMP 4.0 (SIMD) and OpenACC 2.6. Our OpenACC implementation running on NVIDIA’s next-generation Volta GPU boasts an 85.06x speedup over serial code, which is larger than CUDA’s 83.72x speedup over the same serial implementation. We also explore the scalability of our solution using MPI to decompose our simulation domain, allowing us to run on many nodes and accelerators present in state-of-the-art HPC systems. Our parallelization effort across platforms also motivated us to define an abstract parallelism model that is architecture independent, with a goal of creating software abstractions that can be used by applications employing the wavefront sweep motif.

Computer Science↗

A feasibility study on porting the community land model onto accelerators using OpenACC

As environmental models (such as Accelerated Climate Model for Energy (ACME), Parallel Reactive Flow and Transport Model (PFLOTRAN), Arctic Terrestrial Simulator (ATS), etc.) became more and more complicated, we are facing enormous challenges regarding to porting those applications onto hybrid computing architecture. OpenACC appears as a very promising technology, therefore, we have conducted a feasibility analysis on porting the Community Land Model (CLM), a terrestrial ecosystem model within the Community Earth System Models (CESM)). Specifically, we used automatic function testing platform to extract a small computing kernel out of CLM, then we apply this kernel into the actually CLM dataflow procedure, and investigate the strategy of data parallelization and the benefit of data movement provided by current implementation of OpenACC. Even it is a non-intensive kernel, on a single 16-core computing node, the performance (based on the actual computation time using one GPU) of OpenACC implementation is 2.3 time faster than that of OpenMP implementation using single OpenMP thread, but it is 2.8 times slower than the performance of OpenMP implementation using 16 threads. On multiple nodes, MPI_OpenACC implementation demonstrated very good scalability on up to 128 GPUs on 128 computing nodes. This study also provides useful information for us to look into the potential benefits of “deep copy” capability and “routine” feature of OpenACC standards. In conclusion, we believe that our experience on the environmental model, CLM, can be beneficial to many other scientific research programs who are interested to porting their large scale scientific code using OpenACC onto high-end computers, empowered by hybrid computing architecture.

54 ENVIRONMENTAL SCIENCES↗

The PetscSF Scalable Communication Layer

PetscSF, the communication component of the Portable, Extensible Toolkit for Scientific Computation (PETSc), is designed to provide PETSc's communication infrastructure suitable for exascale computers that utilize GPUs and other accelerators. PetscSF provides a simple application programming interface (API) for managing common communication patterns in scientific computations by using a star-forest graph representation. PetscSF supports several implementations based on MPI and NVSHMEM, whose selection is based on the characteristics of the application or the target architecture. An efficient and portable model for network and intra-node communication is essential for implementing large-scale applications. The Message Passing Interface, which has been the de facto standard for distributed memory systems, has developed into a large complex API that does not yet provide high performance on the emerging heterogeneous CPU-GPU-based exascale systems. Here, we discuss the design of PetscSF, how it can overcome some difficulties of working directly with MPI on GPUs, and we demonstrate its performance, scalability, and novel features.

97 MATHEMATICS AND COMPUTING↗

Development of a GPU-based high-performance radiative transfer model for the Infrared Atmospheric Sounding Interferometer (IASI)

Satellite-observed radiance is a nonlinear functional of surface properties and atmospheric temperature and absorbing gas profiles as described by the radiative transfer equation (RTE). In the era of hyperspectral sounders with thousands of high-resolution channels, the computation of the radiative transfer model becomes more time-consuming. The radiative transfer model performance in operational numerical weather prediction systems still limits the number of channels we can use in hyperspectral sounders to only a few hundreds. To take the full advantage of such high-resolution infrared observations, a computationally efficient radiative transfer model is needed to facilitate satellite data assimilation. In recent years the programmable commodity graphics processing unit (GPU) has evolved into a highly parallel, multi-threaded, many-core processor with tremendous computational speed and very high memory bandwidth. The radiative transfer model is very suitable for the GPU implementation to take advantage of the hardware's efficiency and parallelism where radiances of many channels can be calculated in parallel in GPUs. In this paper, we develop a GPU-based high-performance radiative transfer model for the Infrared Atmospheric Sounding Interferometer (IASI) launched in 2006 onboard the first European meteorological polar-orbiting satellites, METOP-A. Each IASI spectrum has 8461 spectral channels. The IASI radiative transfer model consists of three modules. The first module for computing the regression predictors takes less than 0.004% of CPU time, while the second module for transmittance computation and the third module for radiance computation take approximately 92.5% and 7.5%, respectively. Our GPU-based IASI radiative transfer model is developed to run on a low-cost personal supercomputer with four GPUs with total 960 compute cores, delivering near 4 TFlops theoretical peak performance. By massively parallelizing the second and third modules, we reached 364x speedup for 1 GPU and 1455x speedup for all 4 GPUs, both with respect to the original CPU-based single-threaded Fortran code with the -O{sub 2} compiling optimization. The significant 1455x speedup using a computer with four GPUs means that the proposed GPU-based high-performance forward model is able to compute one day's amount of 1,296,000 IASI spectra within nearly 10 min, whereas the original single CPU-based version will impractically take more than 10 days. This model runs over 80% of the theoretical memory bandwidth with asynchronous data transfer. A novel CPU-GPU pipeline implementation of the IASI radiative transfer model is proposed. The GPU-based high-performance IASI radiative transfer model is suitable for the assimilation of the IASI radiance observations into the operational numerical weather forecast model.

97 MATHEMATICS AND COMPUTING↗

Evaluating the Performance of Integer Sum Reduction in SYCL on GPUs

SYCL is a promising programming model for heterogeneous computing—allowing a single-source code to target devices from multiple vendors. One significant task performed on these accelerators is a primitive operation for integer sum reduction. This paper presents several SYCL implementations of integer sum reduction—using atomic functions, shared local memory, vectorized memory accesses and parameterized workload sizes—to compare the performance and maturity of SYCL against open-source vendor-specific implementations of the same reduction. For a sufficiently large number of integers, tuning the parameters of our SYCL implementations achieves 1.4X speedup over the open-source implementations on an Intel UHD630 integrated GPU. The SYCL reduction is 3% faster than the templated reduction in Thrust, and 0.3% faster than the device reduction in CUB on an Nvidia P100 GPU. The SYCL reduction is 1.9% faster than the templated reduction in Thrust, and 0.4% faster than the device reduction in CUB on an Nvidia V100 GPU.

Jin, Zheming↗

Parallel Monte Carlo Methods for Heterogeneous Hardware Computer Systems Using GPUs and Coprocessors: Recent Development of ARCHER Code - Paper 54

Traditional methods of accelerating Monte Carlo (MC) calculations typically use CPUs as the computing unit. The parallelism is based upon multithreading on a multi-core processor, and message passing interface (MPI) across many processors and nodes. In recent years, the hardware accelerators have been introduced by the high performance computing industry as an alternative approach to achieve massive parallelism. Hardware accelerators have the advantage of high Floating-point Operations Per Second (FLOPS) and high energy efficiency. Examples include graphics processing units (GPUs) and coprocessors. Compute-intensive programs can be off-loaded to them to be concurrently executed by hundreds or thousands of active threads. Existing production MC codes, however, cannot be directly compiled or executed on the new platforms. It is therefore necessary to rewrite or redevelop the code using the hardware accelerator specific programming models and APIs. Several studies have been conducted on using GPUs to accelerate MC calculations in reactor analysis and medical physics. In this paper, we present the recent progress at Rensselaer Polytechnic Institute (RPI) in developing a new parallel MC code called ARCHER (Accelerated Radiation-transport Computations in Heterogeneous Environments) for coupled photon-electron transport simulations. The code is designed as a test bed to evaluate the efficacy of heterogeneous computing systems for low-cost MC-based radiation dosimetric calculations. The code has three variants corresponding to three parallel hardware platforms: ARCHER-CPU (multi-core CPU), ARCHER-GPU (Nvidia GPU) and ARCHER-COP (Intel Xeon Phi coprocessor). The code has been used for several applications, including CT imaging dosimetry, radiotherapy dosimetry and radiation shielding design. (authors)

61 RADIATION PROTECTION AND DOSIMETRY↗