Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel codes”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,279 records · Page 71

Simulating Sources of Superstorm Plasmas

We evaluated the contributions to magnetospheric pressure (ring current) of the solar wind, polar wind, auroral wind, and plasmaspheric wind, with the surprising result that the main phase pressure is dominated by plasmaspheric protons. We used global simulation fields from the LFM single fluid ideal MHD model. We embedded the Comprehensive Ring Current Model within it, driven by the LFM transpolar potential, and supplied with plasmas at its boundary including solar wind protons, polar wind protons, auroral wind O+, and plasmaspheric protons. We included auroral outflows and acceleration driven by the LFM ionospheric boundary condition, including parallel ion acceleration driven by upward currents. Our plasmasphere model runs within the CRCM and is driven by it. Ionospheric sources were treated using our Global Ion Kinetics code based on full equations of motion. This treatment neglects inertial loading and pressure exerted by the ionospheric plasmas, and will be superceded by multifluid simulations that include those effects. However, these simulations provide new insights into the respective role of ionospheric sources in storm-time magnetospheric dynamics.

Fok, Mei-Ching↗

Hardware for a real-time multiprocessor simulator

The hardware for a real time multiprocessor simulator (RTMPS) developed at the NASA Lewis Research Center is described. The RTMPS is a multiple microprocessor system used to investigate the application of parallel processing concepts to real time simulation. It is designed to provide flexible data exchange paths between processors by using off the shelf microcomputer boards and minimal customized interfacing. A dedicated operator interface allows easy setup of the simulator and quick interpreting of simulation data. Simulations for the RTMPS are coded in a NASA designed real time multiprocessor language (RTMPL). This language is high level and geared to the multiprocessor environment. A real time multiprocessor operating system (RTMPOS) has also been developed that provides a user friendly operator interface. The RTMPS and supporting software are currently operational and are being evaluated at Lewis. The results of this evaluation will be used to specify the design of an optimized parallel processing system for real time simulation of dynamic systems.

Blech, R. A.↗

Optimizing inference of segmentation on high-resolution images in MLExchange

MLExchange is a machine learning (ML) operations platform providing web user-interfaces (UIs) for data visualization and analysis pipelines at synchrotron facilities. Among these UIs is the segmentation app which helps synchrotron users utilize ML algorithms to automatically segment high-resolution scientific images with minimal manual annotation effort. In this work, we share code optimizations that significantly speed up the segmentation inference workflow of large data in short time. By optimizing the sequence of CPU-GPU data transfers and introducing CPU parallelization to key operations, we improve the per-device, per-image frame computational efficiency and observe close to 3×$$\times$$ speedup over the original segmentation inference workflow run time when utilizing a single GPU. Further adaptations enabling multi-GPU inference yield more than 40×$$\times$$ speedup with 100 GPUs compared to the optimized single GPU inference workflow. This acceleration of the segmentation inference workflow will provide MLExchange users with easy access to segmentation results with little wait time.

Lu, Shizhao↗

Uncovering grain and subgrain microstructure at the scale of additive manufacturing melt tracks with a scalable cellular automaton solidification model

Metal additive manufacturing, characterized by rapid solidification, yields refined grains with a distinctive cellular subgrain microstructure that plays a pivotal role in determining material properties. Due to the significant computational expense demanded to simulate the required physics with submicron spatial resolution, their numerical simulations have been limited to proof-of-concept studies to either 2D or small subregions of a melt pool. In this study, an open-source, scalable, solidification code, muMatScale, based on the cellular automaton method, has been developed to predict the grain and the underlying subgrain microstructure over an entire melt pool. The model incorporates flexible parallelization schemes, utilizing MPI and OpenMP GPU Offloading, in addition to appropriate multi-physics specific to non-equilibrium rapid solidification in AM. The impact of nucleation parameters on grain microstructures was investigated with a focus on grain size variations and morphology transitions. With selected nucleation parameters, the simulation predicted the grain size, subgrain morphology, crystallographic orientation, and microsegregation aligned with experimental measurements. The model demonstrates that epitaxial grain growth is a dominant factor at the melt pool boundary, influencing grain size variation under different grain sizes in the build plate while maintaining consistent primary dendrite arm spacing under identical thermal conditions. Here, the highly efficient numerical model enables large-scale simulations with a spatial resolution of 100 nm or less, unveiling unprecedented insights into thermal and solutal diffusion driven grain growth, and the subgrains with microsegregation within grains in 3D across scales. muMatScale will enable the linking of submicron length-scale microstructure to part-level material behavior by investigating fundamental solidification problems at the intercellular scale in many-track and many-layer builds.

36 MATERIALS SCIENCE↗

Full spectrum optical constant interface to the Materials Project

Optical constants characterize the interaction of materials with light and are important properties in material design. Here we present a Python-based Corvus workflow for simulations of full spectrum optical constants from the visible and ultraviolet to hard x-ray wavelengths based on the real-space Green’s function code FEFF10 and structural data from the Materials Project (MP). The Corvus workflow manager and its associated tools provide an interface to FEFF10 and the MP database. The workflow parallelizes the FEFF computations of optical constants over all absorption edges for each material in the MP database specified by a unique MP-ID. The workflow tools determine the distribution of computational resources needed for that case. Similarly, the optical constants for selected sets of materials can be computed in a single-shot. Additionally, to illustrate the approach, we present results for several elemental solids in the periodic table, as well as a sample compound, and compare our predictions with experimental results. In addition, we provide a database of calculated results for all elements for which there is a stable elemental solid at standard conditions available in the Materials Project database. As in x-ray absorption spectra, these results are interpreted in terms of an atomic-like background and fine-structure contributions.

36 MATERIALS SCIENCE↗

Separatrix-to-Wall Simulations of Impurity Transport with a Fully Three-Dimensional Wall in DIII-D

A novel multi-code workflow to interpret collector probe deposition patterns in DIII-D has been developed. The components of the workflow consist of a detailed computer-aided design (CAD) file of the vessel wall and the scrape-off layer (SOL) codes MAFOT, OSM, DIVIMP and 3DLIM. A special-purpose toolkit enables passing the output of these codes between each other to provide a full-SOL picture of impurity transport. A demonstration of the workflow is described to support evidence of near-SOL tungsten parallel accumulation during trace W impurity experiments on DIII-D. Iteration between simulated deposition patterns in 3DLIM and DIVIMP predicts a region of elevated W density near the separatrix about halfway between the outboard midplane and the top of the plasma. Furthermore, this workflow will be used to better interpret collector probe experiments on DIII-D.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Julienne v1.0.0

Julienne is a compiler-portable unit-testing framework for Fortran software projects, including those that use the parallel/accelerator-programming features of Fortran 2023. Julienne achieves portability across compilers through minimalism and isolation. The minimal design ensures that Julienne uses only features supported by the majority of Fortran compilers. The isolation through zero dependencies ensures that no other projects block Julienne from building with a particular compiler. Julienne also contains with additional services that support its unit-testing code. These include functions for manipulating strings, command lines and input/output format strings; and a user-defined collective subroutine for verifying that all processes pass a test in parallel testing. Julienne's name derives from the term for vegetables sliced into thin strings: julienne vegetables. Julienne captures the authors' most frequently used thin slice of the Veggies and Sourcery software repositories while avoiding certain compiler limitations of the those two packages.

Rouson, Damian↗

Integrating ytopt and libEnsemble to autotune OpenMC

Ytopt is a Python machine-learning-based autotuning software package developed within the ECP PROTEAS-TUNE project. The ytopt software adopts an asynchronous search framework that consists of sampling a small number of input parameter configurations and progressively fitting a surrogate model over the input-output space until exhausting the user-defined maximum number of evaluations or the wall-clock time. libEnsemble is a Python toolkit for coordinating workflows of asynchronous and dynamic ensembles of calculations across massively parallel resources developed within the ECP PETSc/TAO project. libEnsemble helps users take advantage of massively parallel resources to solve design, decision, and inference problems and expands the class of problems that can benefit from increased parallelism. In this paper we present our methodology and framework to integrate ytopt and libEnsemble to take advantage of massively parallel resources to accelerate the autotuning process. Specifically, we focus on using the proposed framework to autotune the ECP ExaSMR application OpenMC, an open source Monte Carlo particle transport code. OpenMC has seven tunable parameters some of which have large ranges such as the number of particles in-flight, which is in the range of 100,000 to 8 million, with its default setting of 1 million. Setting the proper combination of these parameter values to achieve the best performance is extremely time-consuming. Therefore, we apply the proposed framework to autotune the MPI/OpenMP offload version of OpenMC based on a user-defined metric such as the figure of merit (FoM) (particles/s) or energy efficiency energy-delay product (EDP) on Crusher at Oak Ridge Leadership Computing Facility. In conclusion, the experimental results show that we achieve the improvement up to 29.49% in FoM and up to 30.44% in EDP.

Autotuning↗

Assessment of Tools for Molten Salt Reactor Dose Rate Calculations

This report discusses a preliminary assessment of the capabilities of current state-of-the-art stochastic codes Shift and MCNP6 to calculate the ex-core radiation dose rates for a simplified Molten Salt Reactor (MSR) model. The Monte Carlo code Shift has been under significant development in recent years at ORNL as part of the CASL program and is now supported by NEAMS. Originally, Shift was developed for LWR ex-core calculations but with dose rate and shielding calculations specifically requested by the NEAMS program’s MSR industry partners, the MSR Application Drivers team was tasked with assessing Shift for non-LWR applications. This was the first application of the Shift code for non-LWRs and the findings can be considered preliminary due to the activities occurring only over a 5-month period. Attractive features of Shift include massive parallelization and advanced automated variance reduction techniques such as CADIS and FW-CADIS.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Use of gamma ray radiation to parallel the plates of a Fabry-Perot interferometer

The use of gamma radiation to parallel the plates of a Fabry-Perot etalon is examined. The method for determining the etalon parallelism, and the procedure for irradiating the posts are described. Changes in effective gap for the etalon over the surface are utilized to measure the parallelism of the Fabry-Perot etalon. An example in which this technique is applied to an etalon of fused silica plates, which are 132 mm in diameter and coded with zinc sulfide and cryolite, with Zerodur spaces 2 cm in length. The effect of the irradiation of the posts on the thermal performance of the etalon is investigated.

Skinner, Wilbert R.↗

User's and test case manual for FEMATS

The FEMATS program incorporates first-order edge-based finite elements and vector absorbing boundary conditions into the scattered field formulation for computation of the scattering from three-dimensional geometries. The code has been validated extensively for a large class of geometries containing inhomogeneities and satisfying transition conditions. For geometries that are too large for the workstation environment, the FEMATS code has been optimized to run on various supercomputers. Currently, FEMATS has been configured to run on the HP 9000 workstation, vectorized for the Cray Y-MP, and parallelized to run on the Kendall Square Research (KSR) architecture and the Intel Paragon.

Chatterjee, Arindam↗

Performance Analysis and Portability of the PLUM Load Balancing System

The ability to dynamically adapt an unstructured mesh is a powerful tool for solving computational problems with evolving physical features; however, an efficient parallel implementation is rather difficult. To address this problem, we have developed PLUM, an automatic portable framework for performing adaptive numerical computations in a message-passing environment. PLUM requires that all data be globally redistributed after each mesh adaption to achieve load balance. We present an algorithm for minimizing this remapping overhead by guaranteeing an optimal processor reassignment. We also show that the data redistribution cost can be significantly reduced by applying our heuristic processor reassignment algorithm to the default mapping of the parallel partitioner. Portability is examined by comparing performance on a SP2, an Origin2000, and a T3E. Results show that PLUM can be successfully ported to different platforms without any code modifications.

Oliker, Leonid↗

Enhanced Collisional Losses from a Magnetic Mirror Using the Lenard-Bernstein Collision Operator

Collisions are crucial in governing particle and energy transport in plasmas confined in a magnetic mirror trap. Modern gyrokinetic codes model transport in magnetic mirrors, but some utilize approximate model collision operators. This study focuses on a Pastukhov-style method of images calculation of particle and energy confinement times using a Lenard-Bernstein model collision operator. Prior work on parallel particle and energy balances used a different Fokker-Planck plasma collision operator. The method must be extended in non-trivial ways to study the Lenard-Bernstein operator. To assess the effectiveness of our approach, we compare our results with a modern finite element solver. Our findings reveal that the particle confinement time scales like a exp( a 2 ) using the Lenard-Bernstein operator, in contrast to the more accurate scaling that the Coulomb collision operator would yield a 2 exp( a 2 ), where a 2 is approximately proportional to the ambipolar potential. We propose that codes solving for collisional losses in magnetic mirrors utilizing the Lenard-Bernstein or Dougherty collision operator scale their collision frequency of any electrostatically confined species. This study illuminates the collision operator’s intricate role in the Pastukhov-style method of images calculation of collisional confinement.

fusion plasma↗

Recent Developments in the Code RITRACKS (Relativistic Ion Tracks)

The code RITRACKS (Relativistic Ion Tracks) was developed to simulate detailed stochastic radiation track structures of ions of different types and energies. Many new capabilities were added to the code during the recent years. Several options were added to specify the times at which the tracks appear in the irradiated volume, allowing the simulation of dose-rate effects. The code has been used to simulate energy deposition in several targets: spherical, ellipsoidal and cylindrical. More recently, density changes as well as a spherical shell were implemented for spherical targets, in order to simulate energy deposition in walled tissue equivalent proportional counters. RITRACKS is used as a part of the new program BDSTracks (Biological Damage by Stochastic Tracks) to simulate several types of chromosome aberrations in various irradiation conditions. The simulation of damage to various DNA structures (linear and chromatin fiber) by direct and indirect effects has been improved and is ongoing. Many improvements were also made to the graphic user interface (GUI), including the addition of several labels allowing changes of units. A new GUI has been added to display the electron ejection vectors. The parallel calculation capabilities, notably the pre- and post-simulation processing on Windows and Linux machines have been reviewed to make them more portable between different systems. The calculation part is currently maintained in an Atlassian Stash® repository for code tracking and possibly future collaboration.

Plante, Ianik↗

BEYONDPLANCK III. Commander3

We describe the computational infrastructure for end-to-end Bayesian cosmic microwave background (CMB) analysis implemented by the BeyondPlanck Collaboration. The code is called Commander3. It provides a statistically consistent framework for global analysis of CMB and microwave observations and may be useful for a wide range of legacy, current, and future experiments. The paper has three main goals. Firstly, we provide a high-level overview of the existing code base, aiming to guide readers who wish to extend and adapt the code according to their own needs or re-implement it from scratch in a different programming language. Secondly, we discuss some critical computational challenges that arise within any global CMB analysis framework, for instance in-memory compression of time-ordered data, fast Fourier transform optimization, and parallelization and load-balancing. Thirdly, we quantify the CPU and RAM requirements for the current BEYONDPLANCK analysis, finding that a total of 1.5 TB of RAM is required for efficient analysis and that the total cost of a full Gibbs sample for LFI is 170 CPU-hrs, including both low-level processing and high-level component separation, which is well within the capabilities of current low-cost computing facilities. The existing code base is made publicly available under a GNU General Public Library (GPL) license.

79 ASTRONOMY AND ASTROPHYSICS↗

Tokamak ITG-KBM transition benchmarking with the mixed variables/pullback transformation electromagnetic gyrokinetic scheme

Electromagnetic gyrokinetic simulation of high temperature plasma is required to predict confinement in magnetic fusion devices and has posed challenges for existing codes. In this paper, we demonstrate successful global gyrokinetic simulation of the ion temperature gradient-driven mode-kinetic ballooning mode transition in a toroidal fusion plasma test case using the mixed variables/pullback transformation (MV/PT) scheme with the particle-in-cell codes XGC and ORB5, and compare to results from a conventional continuum code from the literature. Furthermore, the MV/PT scheme combines explicit time integration with mitigation of the well-known electromagnetic gyrokinetic “cancelation problem.” We calculate eigenmodes in the electrostatic and parallel vector potentials, and find good agreement in growth rate, real frequency, and the normalized plasma pressure of mode transition.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

LibERI—A portable and performant multi-GPU accelerated library for electron repulsion integrals via OpenMP offloading and standard language parallelism

A portable and performant graphics processing unit (GPU)-accelerated library for electron repulsion integral (ERI) evaluation, named LibERI, has been developed and implemented via directive-based (e.g., OpenMP and OpenACC) and standard language parallelism (e.g., Fortran DO CONCURRENT). Offloaded ERIs consist of integrals over low and high contraction s, p, and d functions using the rotated-axis and Rys quadrature methods. GPU codes are factorized based on previous developments with two layers of integral screening and quartet presorting. In this work, the density screening is moved to the GPU to enhance the computational efficacy for large molecular systems. Here, the L-shells in the Pople basis set are also separated into pure S and P shells to increase the ERI homogeneity and reduce atomic operations and the memory footprint. LibERI is compatible with any quantum chemistry drivers supporting the MolSSI Driver Interface. Benchmark calculations of LibERI interfaced with the GAMESS software package were carried out on various GPU architectures and molecular systems. The results show that the LibERI performance is comparable to other state-of-the-art GPU-accelerated codes (e.g., TeraChem and GMSHPC) and, in some cases, outperforms conventionally developed ERI CUDA kernels (e.g., QUICK) while fully maintaining portability.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Case Study of Using Kokkos and SYCLs Performance-Portable Frameworks for Milc-Dslash Benchmark on NVIDIA, AMD and Intel GPUs

Six of the top ten supercomputers in the TOP500 list from June 2021 rely on NVIDIA GPUs to achieve their peak compute bandwidth. With the announcement of Aurora, Frontier, and El Capitan, Intel and AMD have also entered the domain of providing GPUs for scientific computing. A consequence of the increased diversity in the GPU landscape is the emergence of portable programming models such as Kokkos, SYCL, OpenCL, and OpenMP, which allow application developers to maintain a single-source code across a diverse range of hardware architectures. While the portable frameworks try to optimize the compute resource usage on a given architecture, it is the programmers responsibility to expose parallelism in an application that can take advantage of thousands of processing elements available on GPUs. In this paper, we introduce a GPU-friendly parallel implementation of Milc-Dslash that exposes multiple hierarchies of parallelism in the algorithm. Milc-Dslash was designed to serve as a benchmark with highly optimized matrix-vector multiplications to measure the resource utilization on the GPU systems. The parallel hierarchies in the Milc-Dslash algorithm are mapped onto a target hardware using Kokkos and SYCL programming models. We present the performance achieved by Kokkos and SYCL implementations of Milc-Dslash on NVIDIA A100 GPU, AMD MI100 GPU, and Intel Gen9 GPU. Additionally, we compare the Kokkos and SYCL performances with those obtained from the versions written in CUDA and HIP programming models on NVIDIA A100 GPU and AMD MI100 GPU, respectively.

Dufek, Amanda S↗