Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “OpenMP”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

thornado-transport: Anderson- and GPU-accelerated nonlinear solvers for neutrino-matter coupling

Algorithms for neutrino-matter coupling in core-collapse supernovae (CCSNe) are investigated in the context of a spectral two-moment model, which is discretized in space with the discontinuous Galerkin method, integrated in time with implicit-explicit (IMEX) methods, and implemented in the toolkit for high-order neutrino-radiation hydrodynamics (thornado). The model considers electron neutrinos and antineutrinos and tabulated opacities from Bruenn (1985), which includes neutrino-electron scattering and pair processes. The nonlinear system arising from implicit time discretization of the equations governing neutrino-matter coupling is iterated to convergence using Anderson-accelerated fixed-point methods, which avoid formation of Jacobians and inversion of dense linear systems. Numerical experiments show that, for a given tolerance, a nested iteration scheme which aims to reduce opacity evaluations can lower the computational cost. Our initial port to GPUs, using both OpenMP and OpenACC, shows an overall speedup of up to ~ 100× when compared to results using a single CPU core. These results indicate that the algorithms implemented in thornado are well-suited to GPU acceleration.

Laiu, Paul↗

Threaded Multi-Core GEMM with MoA and Cache-Blocking: Preprint

A threaded multi-core implementation of the high performance dense linear algebra matrix-matrix multiply GEMM kernel is described. This kernel is widely implemented by vendors in the basic linear algebra subroutine BLAS library. The mathematics of arrays (MoA) paradigm due to Mullin (1988) results in contiguous memory accesses by employing outer-product forms. Our performance studies demonstrate that the MoA implementation of double precision DGEMM combined with optimal cache-blocking strategies results in at least a 25% performance gain on the Intel Xeon Skylake processor over the vendor supplied Intel MKL basic linear algebra libraries. Results are presented for the NREL Eagle supercomputer. The multi-core DGEMM achieves over 100 GigaFlops/sec with eight openMP threads.

cache-blocking↗

Pele: An Exascale-Ready Suite of Combustion Codes

High fidelity simulations of realistic combustion devices are extremely demanding computationally because of the requirements to capture complex fuel chemical decomposition, its intricate interactions with turbulent, often multiphase, flows, and the wide separation of space and time scales between the thin flame and the device boundaries. Software required to carry out such computations tends to be extremely complex, particularly when designed to exploit hardware accelerators, and can be difficult to port and maintain. We present Pele, a performance portable suite of tools for the simulation of combustion systems, including codes to evolve reactive multiphase configurations in the low Mach number and compressible flow regimes, along with a set of inter-compatible post processing and in situ analysis tools. The Pele suite of tools is built on top of the AMReX framework for block-structured adaptive mesh refinement, which provides efficient data structures and algorithms that enable the development of a wide variety of efficient mesh and particle based PDE integration schemes. A hierarchical MPI+X parallelism scheme supports CPU-only and accelerated architectures, where X can be OpenMP, CUDA, and HIP based approaches for intra-node computational work distribution. The algorithms and data structures underlying the Pele simulation and analysis tools are highly scalable and performant across a wide variety of high-performance computing platforms, including DOEs newest exascale-class machines, Frontier and Aurora. The simulation and analysis tools are fully documented and freely distributed as open source via GitHub. We present key algorithmic and software challenges, solution strategies, performance and resulting set of capabilities.

AMReX↗

Evaluating Portable Parallelization Strategies for Heterogeneous Architectures in High Energy Physics

High-energy physics (HEP) experiments have developed millions of lines of code over decades that are optimized to run on traditional x86 CPU systems. However, we are seeing a rapidly increasing fraction of floating point computing power in leadership-class computing facilities and traditional data centers coming from new accelerator architectures, such as GPUs. HEP experiments are now faced with the untenable prospect of rewriting millions of lines of x86 CPU code, for the increasingly dominant architectures found in these computational accelerators. This task is made more challenging by the architecture-specific languages and APIs promoted by manufacturers such as NVIDIA, Intel and AMD. Producing multiple, architecture-specific implementations is not a viable scenario, given the available person power and code maintenance issues. The Portable Parallelization Strategies team of the HEP Center for Computational Excellence is investigating the use of Kokkos, SYCL, OpenMP, std::execution::parallel and alpaka as potential portability solutions that promise to execute on multiple architectures from the same source code, using representative use cases from major HEP experiments, including the DUNE experiment of the Long Baseline Neutrino Facility, and the ATLAS and CMS experiments of the Large Hadron Collider. This cross-cutting evaluation of portability solutions using real applications will help inform and guide the HEP community when choosing their software and hardware suites for the next generation of experimental frameworks. We present the outcomes of our studies, including performance metrics, porting challenges, API evaluations, and build system integration.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

MOCUM: A two-dimensional method of characteristics code based on unstructured meshing for general geometries

A transport theory code MOCUM based on the Method of Characteristics (MOC) as the flux solver with an advanced general geometry processor is developed for two-dimensional lattice and full core neutronics modeling. The core structure is represented by Constructive Solid Geometry (CSG) that uses Boolean operations to build complex geometries from simple polygons. Arbitrary-precision arithmetic is also used in the process of building CSG objects to eliminate the round-off error from the commonly used double precision numbers. Then, the constructed core frame will be decomposed and refined into a conforming Delaunay triangulation to ensure the quality of the meshes. The MOC solver kernel is fully paralleled using OpenMP and the developed numerical code is validated by several benchmarks representing various core geometries. For the cases modeled, the maximum percentage error for multiplication factor and the pin power compared to reference values are 0.1% and 0.7% respectively. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Is Monte Carlo embarrassingly parallel?

Monte Carlo is often stated as being embarrassingly parallel. However, running a Monte Carlo calculation, especially a reactor criticality calculation, in parallel using tens of processors shows a serious limitation in speedup and the execution time may even increase beyond a certain number of processors. In this paper the main causes of the loss of efficiency when using many processors are analyzed using a simple Monte Carlo program for criticality. The basic mechanism for parallel execution is MPI. One of the bottlenecks turn out to be the rendez-vous points in the parallel calculation used for synchronization and exchange of data between processors. This happens at least at the end of each cycle for fission source generation in order to collect the full fission source distribution for the next cycle and to estimate the effective multiplication factor, which is not only part of the requested results, but also input to the next cycle for population control. Basic improvements to overcome this limitation are suggested and tested. Also other time losses in the parallel calculation are identified. Moreover, the threading mechanism, which allows the parallel execution of tasks based on shared memory using OpenMP, is analyzed in detail. Recommendations are given to get the maximum efficiency out of a parallel Monte Carlo calculation. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Research on Fast-Doppler-Broadening of neutron cross sections

A Fast-Doppler-Broadening method is developed in this work to broaden Continuous Energy neutron cross-sections for Monte Carlo calculations. Gauss integration algorithm and parallel computing are implemented in this method, which is unprecedented in the history of cross section processing. Compared to the traditional code (NJOY, SIGMA1, etc.), the new Fast-Doppler-Broadening method shows a remarkable speedup with keeping accuracy. The purpose of using Gauss integration is to avoid complex derivation of traditional broadening formula and heavy load of computing complementary error function that slows down the Doppler broadening process. The OpenMP environment is utilized in parallel computing which can take full advantage of modern multi-processor computers. Combination of the two can reduce processing time of main actinides (such as {sup 238}U, {sup 235}U) to an order of magnitude of 1{approx}2 seconds. This new method is fast enough to be applied to Online Doppler broadening. It can be combined or coupled with Monte Carlo transport code to solve temperature dependent problems and neutronics-thermal hydraulics coupled scheme which is a big challenge for the conventional NJOY-MCNP system. Examples are shown to determine the efficiency and relative errors compared with the NJOY results. A Godiva Benchmark is also used in order to test the ACE libraries produced by the new method. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

ABWR start-up test analysis using BWR core simulator with three-dimensional direct response matrix method

The ABWR start-up test analysis has been done with the BWR core simulator using the three--dimensional direct response matrix (3D-DRM) method. The Monte Carlo code VMONT made the sub-response matrices for the 3D-DRM method. Each boundary surface was subdivided by 4 x 4 for transverse segments, by 4 for angular segments and by 4 for axial zones in a node. For the calculation speedup, the 3D-DRM code used the divided sub-response matrices data set. The code used the MPI and OpenMP for the parallelized method. The median value is set as the average critical eigenvalues. The changes from the maximum value to the minimum value are 0.34 %{Delta}k with the spectral history method and 0.40 %{Delta}k without it, and the respective standard deviations were 0.12 % and 0.14 %. Using the spectral history method decreased the variation by 0.06 %{Delta}k. The root mean square differences of the axial power distribution were about 6 % between the analysis results and the plant data. Using the currents which converged in the previous exposure step reduced the number of iterations when the CR pattern changed only slightly. The averaged calculation time for each exposure step was about 5 hours on 12 PC Linux cluster servers with Core 2 Quad 3 GHz. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Parallel 3-D method of characteristics in MPACT

A new parallel 3-D MOC kernel has been developed and implemented in MPACT which makes use of the modular ray tracing technique to reduce computational requirements and to facilitate parallel decomposition. The parallel model makes use of both distributed and shared memory parallelism which are implemented with the MPI and OpenMP standards, respectively. The kernel is capable of parallel decomposition of problems in space, angle, and by characteristic rays up to 0(104) processors. Initial verification of the parallel 3-D MOC kernel was performed using the Takeda 3-D transport benchmark problems. The eigenvalues computed by MPACT are within the statistical uncertainty of the benchmark reference and agree well with the averages of other participants. The MPACT k{sub eff} differs from the benchmark results for rodded and un-rodded cases by 11 and -40 pcm, respectively. The calculations were performed for various numbers of processors and parallel decompositions up to 15625 processors; all producing the same result at convergence. The parallel efficiency of the worst case was 60%, while very good efficiency (>95%) was observed for cases using 500 processors. The overall run time for the 500 processor case was 231 seconds and 19 seconds for the case with 15625 processors. Ongoing work is focused on developing theoretical performance models and the implementation of acceleration techniques to minimize the number of iterations to converge. (authors)

97 MATHEMATICS AND COMPUTING↗

Progress and status of the OpenMC Monte Carlo code

The present work describes the latest advances and progress in the development of the OpenMC Monte Carlo code, an open-source code originating from the Massachusetts Institute of Technology. First, an overview of the development workflow of OpenMC is given. Various enhancements to the code such as real-time XML input validation, state points, plotting, OpenMP threading, and coarse mesh finite difference acceleration are described. (authors)

97 MATHEMATICS AND COMPUTING↗

Recent developments in DYNSUB: New models, code optimization and parallelization

DYNSUB is a high-fidelity coupled code system consisting of the reactor simulator DYN3D and the sub-channel code SUBCHANFLOW. It describes nuclear reactor core behavior with pin-by-pin resolution for both steady-state and transient scenarios. In the course of the coupled code system's active development, super-homogenization (SPH) and generalized equivalence theory (GET) discontinuity factors may be computed with and employed in DYNSUB to compensate pin level homogenization errors. Because of the largely increased numerical problem size for pin-by-pin simulations, DYNSUB has bene fitted from HPC techniques to improve its numerical performance. DYNSUB's coupling scheme has been structurally revised. Computational bottlenecks have been identified and parallelized for shared memory systems using OpenMP. Comparing the elapsed time for simulating a PWR core with one-eighth symmetry under hot zero power conditions applying the original and the optimized DYNSUB using 8 cores, overall speed up factors greater than 10 have been observed. The corresponding reduction in execution time enables a routine application of DYNSUB to study pin level safety parameters for engineering sized cases in a scientific environment. (authors)

97 MATHEMATICS AND COMPUTING↗

Scalable Graph Analytics and HPC Operational Enhancement: Parallel Computing and ML/DL Innovations

Parallel computing plays a pivotal role in the efficient processing of large-scale graphs. Complex network analysis stands as a capti- vating research frontier, holding promise across diverse scientific domains such as sociology, biology, online media, and recommenda- tion systems. In this era, Machine Learning (ML) and Deep Learning (DL) have emerged as indispensable tools, underpinning remarkable technological achievements. Within this dynamic landscape, my research revolves around advancing parallel algorithms tailored for large-scale graph operations. To achieve this, I harness the power of cutting-edge technologies including OpenMP, MPI, HIP, and CUDA, on the High-Performance Computing (HPC) platforms to unlock optimal performance. I also apply ML/DL techniques to HPC operational data, to streamline the monitoring and maintenance of supercomputers, alleviating the complexities associated with their upkeep and enhancing user support. My research echoes the syn- ergy between parallel computing, large-scale graph analysis, and ML/DL, improving computational efficiency and user experience.

Sattar, Naw Safrin↗

Pin-by-Pin Core Calculation with an NEM-Based Two-Level Hybrid CMFD Algorithm

In the near future, systematic pin-by-pin core calculations are expected to provide a high level of accuracy. These are being developed in pursuit of high-fidelity tools for analyzing the reactor configurations. One such tool that has recently gained significant attention is the pin-by-pin core calculation via local-global iteration using nonlinear acceleration scheme, such as the One-Node CMFD (Coarse-Mesh Finite Difference) nonlinear algorithm. The One-Node CMFD method uses two correction factors to preserve the interface current, in contrast to the one correction term employed by conventional nonlinear iterative methods such as that proposed by Kord. S. Smith. These correction factors are updated with surface flux and net current which are obtained from a high order local problem. With two correction factors, One-Node CMFD can reduce computing time of local FMFD (Fine-Mesh Finite Difference) as it nonlinearly couples the CMFD and FMFD methods in global-local iterations. Nonetheless, very small mesh size is still required and it consequently requires long computing time for high accuracy. As such, a more flexible approach is needed to address this challenge. This paper presents one such possible approach named the hCMFD (Hybrid CMFD) method. In this method, One-Node CMFD is used to solve the global problem, while Two-Node NEM (Nodal Expansion Method) CMFD replaces FMFD as the solver for the local problem. In this paper, two kinds of CMFD methods has been simultaneously implemented for a global-local nonlinear iteration in order to do pin-by-pin heterogeneous reactor calculations. The global problem is solved by a one-node CMFD which uses two correction factors, while the local fixed-source problem is solved by the conventional two-node NEM CMFD which use one correction factor on a surface. From solution of the global eigenvalue problem, incoming partial current is obtained and it is used as boundary condition of the local fixed-source problems. It has been shown that over 99% of the whole calculations can be done in parallel with the new CMFD method for 2-D problems and the algorithm can be easily parallelized by using the OpenMP parallel scheme. The hCMFD method will enable an efficient pin-by-pin reactor analysis since a very efficient parallel calculation is possible with the hCMFD method. In future works, the hCMFD method will be extended to three-dimensional reactor problem. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Reductive Analysis with Compiler-Guided Large Language Models for Input-Centric Code Optimizations

Input-centric program optimization aims to optimize code by considering the relations between program inputs and program behaviors. Despite its promise, a long-standing barrier for its adoption is the difficulty of automatically identifying critical features of complex inputs. This paper introduces a novel technique, reductive analysis through compiler-guided Large Language Models (LLMs), to solve the problem through a synergy between compilers and LLMs. It uses a reductive approach to overcome the scalability and other limitations of LLMs in program code analysis. The solution, for the first time, automates the identification of critical input features without heavy instrumentation or profiling, cutting the time needed for input identification by 44× (or 450× for local LLMs), reduced from 9.6 hours to 13 minutes (with remote LLMs) or 77 seconds (with local LLMs) on average, making input characterization possible to be integrated into the workflow of program compilations. Optimizations on those identified input features show similar or even better results than those identified by previous profiling-based methods, leading to optimizations that yield 92.6% accuracy in selecting the appropriate adaptive OpenMP parallelization decisions, and 20-30% performance improvement of serverless computing while reducing resource usage by 50-60%.

Input-Centric Optimization↗

Performance Evaluation of Different Parallel Programming Models in SCALE-Shift Sequences for Criticality and Shielding Applications [Abstract]

The SCALE code system has been widely used for nuclear criticality safety, reactor physics, radiation shielding, source term generation, and inventory analyses by researchers, industry, and regulatory bodies. Although limited support for shared- and distributed-memory parallel processing was introduced via C++ threading, OpenMP, and MPI, a hybrid parallel programming model with both distributed- and shared-memory parallelism has not been fully supported in the SCALE code system.

Nuclear Criticality Safety Program (NCSP)↗

Evaluating Application Characteristics for GPU Portability Layer Selection

GPUs have become the dominant source of computing power for high performance computing and are increasingly being used across the High Energy Physics computing landscape for a wide variety of tasks. Though NVIDIA is currently the main provider of GPUs, AMD and Intel are rapidly increasing their market share. As a result, programming using a vendor-specific language such as CUDA can significantly reduce deployment choices. There are a number of portability layers such as Kokkos, Alpaka, SYCL, OpenMP and std::par that permit execution on a broad range of GPU and CPU architectures, significantly increasing the flexibility of application programmers. However, each of these portability layers has its own characteristics, performing better at some tasks and worse at others, or placing limitations on aspects of the application. In this presentation, we report on a study of application and kernel characteristics that can influence the choice of a portability layer and show how each layer handles these characteristics. We have analyzed representative heterogeneous applications from CMS (patatrack and p2r), DUNE (Wire-Cell Toolkit), and ATLAS (FastCaloSim) to identify key application characteristics that have different behaviors for the various portability technologies. Using these results, developers can make more informed decisions on which GPU portability technology is best suited to their application.

Atif, Mohammad [Brookhaven]↗

Evaluation of LLVM Flang for Production HPC Applications and Modern Fortran Features

In 2025, LLVM released its first Flang Fortran compiler version considered ready for widespread evaluation. We know of no published assessment of Flang compiling a workload- derived portfolio of high-performance computing (HPC) applications. We address this gap using workload data from the National Energy Research Scientific Computing Center (NERSC), which supports more than 10,000 scientists on approximately 1,000 projects. The NERSC workload analyses identify many Fortran components in heavily used applications. We selected 10 such packages with available source code. We compiled them with Flang 22.1.3 on NERSC’s Perlmutter system. Six compiled without code modifications, though some required build-system changes. Three compiled after minor source edits, mostly to address Fortran standard violations. One built only without OpenMP enabled. We evaluated seven additional packages selected for their use of, or enablement of, standard Fortran parallel features: multi-image execution and do concurrent. Six such codes compiled with most or all unit tests passing.

Rasmussen, Katherine↗

A hybrid parallel framework for the cellular Potts model simulations

The Cellular Potts Model (CPM) has been widely used for biological simulations. However, most current implementations are either sequential or approximated, which can't be used for large scale complex 3D simulation. In this paper we present a hybrid parallel framework for CPM simulations. The time-consuming POE solving, cell division, and cell reaction operation are distributed to clusters using the Message Passing Interface (MPI). The Monte Carlo lattice update is parallelized on shared-memory SMP system using OpenMP. Because the Monte Carlo lattice update is much faster than the POE solving and SMP systems are more and more common, this hybrid approach achieves good performance and high accuracy at the same time. Based on the parallel Cellular Potts Model, we studied the avascular tumor growth using a multiscale model. The application and performance analysis show that the hybrid parallel framework is quite efficient. The hybrid parallel CPM can be used for the large scale simulation ({approx}10{sup 8} sites) of complex collective behavior of numerous cells ({approx}10{sup 6}).

97 MATHEMATICS AND COMPUTING↗