Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Computer performance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

The Benefits and Weaknesses of Containerizing Software for HPC

Containerization technology has emerged as a transformative tool for software engineers, offering consistent development and deployment environments, simplifying dependency management, and enhancing scalability and portability across diverse systems. However, its application in High-Performance Computing (HPC) presents unique challenges, including the management of virtualization overhead, the need for efficient resource allocation, and the maintenance of optimal performance for compute-intensiv

Ho, Eric Victor [Sandia National Laboratories (SNL↗
DOE OSTI2025

A Global Perspective on Supercomputer Power Provisioning: Case Studies from United States and Europe

Electrical provisioning in high performance computing is transitioning from simple nameplate Thermal Design Power (TDP) models to more nuanced approaches based on expected electrical load. This paper captures current power provisioning strategies across six international supercomputing centers and seven systems, three of which (Lumi, Summit, Sierra) were in the top 10 of the Top500 list at the time of data collection1. We present longitudinal and summary data of actual power consumption as well as a discussion of how each site approached the question of provisioning. We conclude with a discussion on future directions of hardware overprovisioning and its implications for machine and electrical utilization.

Patki, Tapasya [Lawrence Livermore National Labora↗
DOE OSTI2023

Mastering HPC Runtime Prediction: From Observing Patterns to a Methodological Approach

The continual expansion of high-performance computing (HPC) brings with it an increasing need for efficiency. Heavy investment in energy, hardware, and software infrastructure to support peta- and exascale computing requires the optimization of existing systems and, wherever possible, the discernment and adoption of best-practices towards these goals. Such is the case for runtime prediction. When a job is submitted to an HPC system, an estimate of its runtime is provided by the user in the form of "requested wallclock". Error in this user-provided estimate can lead to jobs being prematurely killed by the scheduler, increased wait time on the queue, and decreased system utilization. More than fifteen years of research has been directed at mitigating these effects by using data-driven runtime predictions. Codified here is a set of commonalities and insights emerging from this body of work, which we present as recommendations and best practices. These practices are combined into a methodological approach described and evaluated on an 11-million-job dataset from the National Renewable Energy Laboratory's petascale HPC system, Eagle. This dataset and the accompanying codebase have been released to the public domain for the benefit of the wider HPC research community.

high performance computing↗
DOE OSTI2023

Mastering HPC Runtime Prediction: From Observing Patterns to a Methodological Approach: Preprint

The continual expansion of high-performance computing (HPC) brings with it an increasing need for efficiency. Heavy investment in energy, hardware, and software infrastructure to support peta- and exascale computing requires the optimization of existing systems and, wherever possible, the discernment and adoption of best-practices towards these goals. Such is the case for runtime prediction. When a job is submitted to an HPC system, an estimate of its runtime is provided by the user in the form of "requested wallclock''. Error in this user-provided estimate can lead to jobs being prematurely killed by the scheduler, increased wait time on the queue, and decreased system utilization. More than fifteen years of research has been directed at mitigating these effects by using data-driven runtime predictions. Codified here is a set of commonalities and insights emerging from this body of work, which we present as recommendations and best practices. These practices are combined into a methodological approach described and evaluated on an 11-million-job dataset from the National Renewable Energy Laboratory's petascale HPC system, Eagle. This dataset and the accompanying codebase have been released to the public domain for the benefit of the wider HPC research community.

high performance computing↗
DOE OSTI2025

ERF: Energy Research and Forecasting Model

High performance computing (HPC) architectures have undergone rapid development in recent years. As a result, established software suites face an ever increasing challenge to remain performant on and portable across modern systems. Many of the widely adopted atmospheric modeling codes cannot fully (or in some cases, at all) leverage the acceleration provided by General-Purpose Graphics Processing Units, leaving users of those codes constrained to increasingly limited HPC resources. Energy Research and Forecasting (ERF) is a regional atmospheric modeling code that leverages the latest HPC architectures, whether composed of only Central Processing Units (CPUs) or incorporating GPUs. ERF contains many of the standard discretizations and basic features needed to model general atmospheric dynamics. The modular design of ERF provides a flexible platform for exploring different physics parameterizations and numerical strategies. ERF is built on a state-of-the-art, well-supported, software framework (AMReX) that provides a performance portable interface and ensures ERF's long-term sustainability on next generation computing systems. This paper details the numerical methodology of ERF, presents results for a series of verification/validation cases, and documents ERF's performance on current HPC systems. The roughly 5× speed up of ERF (using GPUs) over Weather Research and Forecasting (CPUs only) for a 3D squall line test case highlights the significance of leveraging GPU acceleration.

17 WIND ENERGY↗
DOE OSTI2019

Determining the impact of a severe dry to wet transition on watershed hydrodynamics in California, USA with an integrated hydrologic model

With the onset of climate change, regions relied upon for water supply are increasingly subject to end-member fluctuations between periods of severe drought followed by extreme precipitation. The impacts of these extreme conditions on watershed hydrodynamics in water-resource sensitive regions such as California are unknown despite their great importance for resilience and water management purposes. Understanding these impacts requires high-resolution physically based models to capture sharp variations of topography, land use, wetting fronts, etc. Here, an integrated hydrologic model was used in a high-performance computing framework to study the complex nonlinear dynamics occurring at a representative Californian watershed. The Cosumnes Watershed, one of the last major rivers in California without a dam, offers a rare opportunity to isolate the effects of water management from climate extremes. Here, we show model validation with comparisons between model outputs and local measurements in addition to various satellite-based products including (1) Snow Water Equivalent (SWE) with Snow Data Assimilation System (SNODAS) and a reconstruction method by Bair and co-authors, (2) soil moisture with Soil Moisture Active Passive (SMAP), and (3) evapotranspiration (ET) with Mapping Evapotranspiration at high Resolution with Internalized Calibration (METRIC). To assess changes in hydrodynamic behavior during climate extremes and their transitions, a simulation spanning a recent drought followed by the highest precipitation year on record (2015–2017) is discussed. From these simulations, we are able to highlight regions that are the most sensitive to climate extremes, which depend on many factors including hydrologic connectivity, geology and topography. These analyses provide a better understanding of the physical phenomena occurring in the watershed, strengthening our knowledge of how the system may respond to extreme conditions which might become the “new normal.”

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Accelerating Relative-error Bounded Lossy Compression for HPC datasets with Precomputation-Based Mechanisms

Scientific simulations in high-performance computing (HPC) environments are producing vast volume of data, which may cause a severe I/O bottleneck at runtime and a huge burden on storage space for post-analysis. Unlike the traditional data reduction schemes (such as deduplication or lossless compression), not only can error-controlled lossy compression significantly reduce the data size but it can also hold the promise to satisfy user demand on error control. Point-wise relative error bounds (i.e., compression errors depends on the data values) are widely used by many scientific applications in the lossy compression, since error control can adapt to the precision in the dataset automatically. Point-wise relative error bounded compression is complicated and time consuming. In this work, we develop efficient precomputation-based mechanisms in the SZ lossy compression framework. Our mechanisms can avoid costly logarithmic transformation and identify quantization factor values via a fast table lookup, greatly accelerating the relative-error bounded compression with excellent compression ratios. In addition, our mechanisms also help reduce traversing operations for Huffman decoding, and thus significantly accelerate the decompression process in SZ. Experiments with four well-known real-world scientific simulation datasets show that our solution can improve the compression rate by about 30% and decompression rate by about 70% in most of cases, making our designed lossy compression strategy the best choice in class in most cases.

compression rate↗
DOE OSTI2026

State of the art, gaps, and prospects in fusion materials theory and modelling

Advancing the theory and simulation of materials for fusion applications remains a key component of global roadmaps aimed at delivering much-needed fusion power. Especially as the drive for commercial application increases, prototypes must be designed against radiation damage before the relevant experimental data can be collected and cost reductions that are possible by testing materials in silico become even more important. Here, we summarise the state of the art as it emerged during the 7 th Fusion Materials Theory & Modelling Workshop that took place in 2024, with the aim to highlight present gaps and future directions for the fusion materials modelling community. Of particular interest were the effects of transmutations, chemical complexity with the development of novel alloys and interatomic potentials, advancements in modelling high-dose microstructures, comparison with experimental data and multiscale models for structural assessment relying on high-performance computing and virtual reality.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Height Above Nearest Drainage (HAND) and Hydraulic Property Table for CONUS - Version 0.21 (20200601)

The continental flood inundation mapping (CFIM) framework is a high-performance computing (HPC)-based computational framework for the Height Above Nearest Drainage (HAND)-based inundation mapping methodology. Using the 10m Digital Elevation Model (DEM) data produced by U.S. Geological Survey (USGS) 3DEP (the 3-D Elevation Program) and the NHDPlus hydrography dataset produced by USGS and the U.S. Environmental Protection Agency (EPA), a hydrological terrain raster called HAND is computed for HUC6 units in the conterminous U.S. (CONUS). The value of each raster cell in HAND is an approximation of the relative elevation between the cell and its nearest water stream. Derived from HAND, a hydraulic property table is established to calculate river geometry properties for each of the 2.7 million river reaches covered by NHDPlus (5.5 million kilometers in total length). This table is a lookup table for water depth given an input stream flow value. Such lookup is available between water depth 0m and 25m at 1-foot interval. The flood inundation map is then computed by using HAND and this lookup table based on the near real-time water forecast from the National Water Model (NWM) at the National Oceanic and Atmospheric Administration (NOAA). HAND and the Hydraulic Property Table version v0.21 is an upgrade to v0.2. Version 0.21 applies NHD HR etching to DEMs for all HUC6 units before running the HAND workflow. This new version also fixed the issue of void areas along HUC boundary. This dataset comprises 331 HUC6 units for CONUS (excluding the five great lakes units), each is a downloadable zip file. Version 0.1 was computed at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urban-Champaign in 2016 and is currently hosted at the Texas Advanced Computing Center (TACC). Please cite this data DOI and the following publications and data DOIs when you use HAND and the hydraulic property table: Liu, Yan Y. Height Above Nearest Drainage (HAND) and Hydraulic Property Table for CONUS - Version 0.2. (20200301). Oak Ridge Leadership Computing Facility. (2020). DOI: 10.13139/ORNLNCCS/1608331 Liu, Yan Y., David R. Maidment, David G. Tarboton, Xing Zheng, and Shaowen Wang. 'A CyberGIS integration and computation framework for high-resolution continental-scale flood inundation mapping.' JAWRA Journal of the American Water Resources Association 54, no. 4 (2018): 770-784. DOI: 10.1111/1752-1688.12660 Zheng, Xing, David G. Tarboton, David R. Maidment, Yan Y. Liu, and Paola Passalacqua. 'River channel geometry and rating curve estimation using height above the nearest drainage.' JAWRA Journal of the American Water Resources Association 54, no. 4 (2018): 785-806. DOI: 10.1111/1752-1688.12661 Liu, Y. (2018). Height Above Nearest Drainage (HAND) for CONUS, HydroShare, https://doi.org/10.4211/hs.69f7d237675c4c73938481904358c789 LICENSE FOR USE -- MAPS AND DATA DISCLAIMER This resource is shared under the Creative Commons Attribution CC BY, http://creativecommons.org/licenses/by/4.0/ MAPS AND DATA DISCLAIMER The Oak Ridge National Laboratory (ORNL) shall not be held liable for improper or incorrect use of the data described or information contained on this map or associated series of maps. The data and related map graphics are not legal, land survey or engineering documents and are not intended to be used as such. ORNL gives no warranty, express or implied, as to the accuracy, reliability, utility or completeness of this information. The user of these maps and data assumes all responsibility and risk for the use of the maps and data. ORNL disclaims all warranties, representations or endorsements either express or implied, with regard to the information contained in this map product, including, but not limited to, all implied warranties of merchantability, fitness for a particular purpose or non-infringement. This preliminary map product is for research and review purposes only. It is not intended to be used for emergency management operational or life safety decisions at the local or regional governmental level or by the general public. Users requiring information regarding hazardous conditions or meteorological conditions for specific geographic areas should consult directly with their city or county emergency management office.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Height Above Nearest Drainage (HAND) and Hydraulic Property Table for CONUS

The continental flood inundation mapping (CFIM) framework is a high-performance computing (HPC)-based computational framework for the Height Above Nearest Drainage (HAND)-based inundation mapping methodology. Using the 10m Digital Elevation Model (DEM) data produced by U.S. Geological Survey (USGS) 3DEP (the 3-D Elevation Program) and the NHDPlus hydrography dataset produced by USGS and the U.S. Environmental Protection Agency (EPA), a hydrological terrain raster called HAND is computed for HUC6 units in the conterminous U.S. (CONUS). The value of each raster cell in HAND is an approximation of the relative elevation between the cell and its nearest water stream. Derived from HAND, a hydraulic property table is established to calculate river geometry properties for each of the 2.7 million river reaches covered by NHDPlus (5.5 million kilometers in total length). This table is a lookup table for water depth given an input stream flow value. Such lookup is available between water depth 0m and 25m at 1-foot interval. The flood inundation map is then computed by using HAND and this lookup table based on the near real-time water forecast from the National Water Model (NWM) at the National Oceanic and Atmospheric Administration (NOAA). HAND and the Hydraulic Property Table version 0.2. is created to correspond to data updates in the USGS 1/3 arcsec DEM, the USGS National Hydrography Dataset, including its Water Boundary Dataset, and the NHDPlus medium resolution dataset. This dataset comprises 331 HUC6 units for CONUS (excluding the five great lakes units), each is a downloadable zip file. Version 0.1 was computed at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urban-Champaign in 2016 and is currently hosted at the Texas Advanced Computing Center (TACC). Please cite this data DOI and the following publications and data DOIs when you use HAND and the hydraulic property table: Liu, Yan Y., David R. Maidment, David G. Tarboton, Xing Zheng, and Shaowen Wang. 'A CyberGIS integration and computation framework for high-resolution continental-scale flood inundation mapping.' JAWRA Journal of the American Water Resources Association 54, no. 4 (2018): 770-784. DOI: 10.1111/1752-1688.12660 Zheng, Xing, David G. Tarboton, David R. Maidment, Yan Y. Liu, and Paola Passalacqua. 'River channel geometry and rating curve estimation using height above the nearest drainage.' JAWRA Journal of the American Water Resources Association 54, no. 4 (2018): 785-806. DOI: 10.1111/1752-1688.12661 Liu, Y. (2018). Height Above Nearest Drainage (HAND) for CONUS - v0.1, HydroShare, https://doi.org/10.4211/hs.69f7d237675c4c73938481904358c789 LICENSE FOR USE -- MAPS AND DATA DISCLAIMER This resource is shared under the Creative Commons Attribution CC BY, http://creativecommons.org/licenses/by/4.0/ MAPS AND DATA DISCLAIMER The Oak Ridge National Laboratory (ORNL) shall not be held liable for improper or incorrect use of the data described or information contained on this map or associated series of maps. The data and related map graphics are not legal, land survey or engineering documents and are not intended to be used as such. ORNL gives no warranty, express or implied, as to the accuracy, reliability, utility or completeness of this information. The user of these maps and data assumes all responsibility and risk for the use of the maps and data. ORNL disclaims all warranties, representations or endorsements either express or implied, with regard to the information contained in this map product, including, but not limited to, all implied warranties of merchantability, fitness for a particular purpose or non-infringement. This preliminary map product is for research and review purposes only. It is not intended to be used for emergency management operational or life safety decisions at the local or regional governmental level or by the general public. Users requiring information regarding hazardous conditions or meteorological conditions for specific geographic areas should consult directly with their city or county emergency management office.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

High-bandwidth density silicon photonic resonators for energy-efficient optical interconnects

The growth of artificial intelligence applications demands ever larger and more complex deep learning models, dominating today's—and tomorrow's—data center and high-performance computing systems. While traditional electronics are failing to keep pace with application demands, silicon photonic (SiPh) interconnects have emerged as a necessary technology to support these systems. SiPh-driven wavelength-division multiplexing (WDM) offers a particularly promising path toward supporting incredibly high-aggregate link bandwidth in a compact and efficient form factor. One of the basic building blocks of these integrated WDM interconnects is the SiPh resonator. Their inherent wavelength selectivity and compact footprint allow for efficient data transmission multiplexed across dozens of carrier wavelengths. Used as add-drop (AD) filters, SiPh resonators are critical to constructing integrated tunable wavelength-selective optical circuit switches as well as for demultiplexing the different carrier wavelengths toward independent wavelength-insensitive photodiodes in a dense wavelength-division multiplexing receiver. Resonators in the all-pass (AP) configuration are widespread as well, allowing for wavelength-selective modulation to drive aggregate link bandwidths far beyond the individual channel data rate. Unlike SiPh Mach–Zehnder modulators (MZM), resonant modulators can be driven using low, complementary metal-oxide-semiconductor drive voltages, allowing for tight co-integration between photonic integrated circuits, fabricated with larger process node technologies, and electronic integrated circuits, designed to exploit the advantages of the latest node. To push toward practical peta-scale interconnects, a comprehensive review of SiPh resonators is required, addressing bottlenecks and design constraints at both the architecture and device levels. We first describe the predominant integrated link architectures and identify their limits. We then discuss the device-level design considerations that can be made for both AD and AP configuration resonators to overcome the system level limits with novel resonator device designs. Analytical models and numerical simulation of resonators are validated by experimental measurement of devices fabricated in a commercial 300-mm foundry, showing a clear path toward volume manufacturing. The demonstrated resonant modulators and filters support the feasibility of increasing the aggregate bandwidth of resonator-driven SiPh interconnects into the peta-scale regime.

42 ENGINEERING↗
DOE OSTI2020

3D Coded SUMMA: Communication-Efficient and Robust Parallel Matrix Multiplication

In this paper, we propose a novel fault-tolerant parallel matrix multiplication algorithm called 3D Coded SUMMA that achieves higher failure-tolerance than replication-based schemes for the same amount of redundancy. This work bridges the gap between recent developments in coded computing and fault-tolerance in high-performance computing (HPC). The core idea of coded computing is the same as algorithm-based fault-tolerance (ABFT), which is weaving redundancy in the computation using error-correcting codes. In particular, we show that MatDot codes, an innovative code construction for parallel matrix multiplications, can be integrated into three-dimensional SUMMA (Scalable Universal Matrix Multiplication Algorithm [30]) in a communication-avoiding manner. To tolerate any two node failures, the proposed 3D Coded SUMMA requires ~50% less redundancy than replication, while the overhead in execution time is only about 5–10%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Randomized low-rank decompositions of nuclear three-body interactions

First-principles simulations of many-fermion systems are commonly limited by the computational requirements of processing large data objects. As a remedy, we propose the use of low-rank approximations of three-body interactions, which are the dominant such limitation in nuclear physics. We introduce a randomized decomposition technique to handle the excessively large matrix dimensions and study the sensitivity of low-rank properties to interaction details. The developed low-rank three-nucleon interactions are benchmarked in ab initio simulations of few- and many-body systems. Exploiting low-rank properties provides a promising route to extend the microscopic description of atomic nuclei to large systems where storage requirements exceed the computational capacities of the most advanced high-performance computing facilities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2016

Benchmark Generation and Simulation at Extreme Scale

The path to extreme scale high-performance computing (HPC) poses several challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Investigating the performance of parallel applications at scale on future architectures and the performance impact of different architectural choices is an important component of HPC hardware/software co-design. Simulations using models of future HPC systems and communication traces from applications running on existing HPC systems can offer an insight into the performance of future architectures. This work targets technology developed for scalable application tracing of communication events. It focuses on extreme-scale simulation of HPC applications and their communication behavior via lightweight parallel discrete event simulation for performance estimation and evaluation. Instead of simply replaying a trace within a simulator, this work promotes the generation of a benchmark from traces. This benchmark is subsequently exposed to simulation using models to reflect the performance characteristics of future-generation HPC systems. This technique provides a number of benefits, such as eliminating the data intensive trace replay and enabling simulations at different scales. The presented work features novel software co-design aspects, combining the ScalaTrace tool to generate scalable trace files, the ScalaBenchGen tool to generate the benchmark, and the xSim tool to assess the benchmark characteristics within a simulator.

Lagadapati, Mahesh↗
DOE OSTI2021

Machine learning-based analysis of COVID-19 pandemic impact on US research networks

Here in this study we explore how fallout from the changing public health policy around COVID-19 has changed how researchers access and process their science experiments. Using a combination of techniques from statistical analysis and machine learning, we conduct a retrospective analysis of historical network data for a period around the stay-at-home orders that took place in March 2020. Our analysis takes data from the entire ESnet infrastructure to explore DOE high-performance computing (HPC) resources at OLCF, ALCF, and NERSC, as well as User sites such as PNNL and JLAB. We look at detecting and quantifying changes in site activity using a combination of t-Distributed Stochastic Neighbor Embedding (t-SNE) and decision tree analysis. Our findings bring insights into the working patterns and impact on data volume movements, particularly during late-night hours and weekends.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A Quantum Analog Coprocessor for Correlated Electron Systems Simulation

Analog quantum simulation is an approach for studying physical systems that might otherwise be computationally intractable to simulate on classical high-performance computing (HPC) systems. The key idea behind analog quantum simulation is the realization of a physical system with a low-energy effective Hamiltonian that is the same as the low-energy effective Hamiltonian of some target system to be studied. Purpose-built nanoelectronic devices are a natural candidate for implementing the analog quantum simulation of strongly correlated materials that are otherwise challenging to study using classical HPC systems. However, realizing devices that are sufficiently large to study the properties of a non-trivial material system (e.g., those described by a Fermi-Hubbard model) will eventually require the fabrication, control, and measurement of at least 0(10) quantum dots, or other engineered quantum impurities. As a step toward large-scale analog or digital quantum simulation platforms based on nanoelectronic devices, we propose a new approach to analog quantum simulation that makes use of the large Hilbert space dimension of the electronic baths that are used to adjust the occupancy of one or a few engineered quantum impurities. This approach to analog quantum simulation allows us to study a wide array of quantum impurity models. We can further augment the computational power of such an approach by combining it with a classical computer to facilitate dynamical mean-field theory (DMFT) calculations. DMFT replaces the solution of a lattice impurity problem with the solution of a family of localized impurity problems with bath couplings that are adjusted to satisfy a self-consistency condition between the two models. In DMFT, the computationally challenging task is the high-accuracy solution of an instance of a quantum impurity model that is determined self-consistently in coordination with a mean-field calculation. We propose using one or a few engineered quantum impurities with adjustable couplings to baths to realize an analog quantum coprocessor that effects the solution of such a model through measurements of a physical quantum impurity, operating in coordination with a classical computer to achieve a self-consistent solution to a DMFT calculation. We focus on implementation details relevant to a number of technologies for which Sandia has design, fabrication, and measurement expertise. The primary technical advances outlined in this report concern the development of a supporting modeling capability. As with all analog quantum simulation platforms, the successful design and operation of individual devices depends critically on one's ability to predict the effective low-energy Hamiltonian governing its dynamics Our project has made this possible and lays the foundation for future experimental implementations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Physics Basis, Optimization, and Control for Integrated 3D Edge Long-pulse Tokamak Scenarios

The aim of this project with international collaborations (PI: J.-K. Park of Princeton Plasma Physics Laboratory) is to develop a unified physics basis and predictive capability for the control of edge-localized modes (ELMs) with optimized non-axisymmetric (3D) fields, by leveraging the unique research capabilities of international KSTAR tokamak in Korea. At the University of California, Irvine, we use a fusion simulation code GTC to study the effects of the 3D magnetic fields (including both magnetic islands and stochastic magnetic field lines) on microturbulence. We focus on the effects of 3D equilibrium on turbulent and neoclassical transport that leads to ELM suppression. During the project period, the UCI team of this collaboration has made significant progress in understanding turbulent transport in 3D geometry including both tokamaks with RMP and stellarators. Four UCI postdoctoral researchers, Drs. Javier Nicolau, Gyungjin Choi, Pengfei Liu, and Xishuo Wei were partially supported by this project and collaborating with the entire project team. We have published 4 journal papers and gave several invited talks. This project advance physics understand of 3D effects on microturbulence, specially effects of magnetic islands. It will contribute to the DOE SciDAC ISEP project in the area of GTC development for general 3D geometry capability, rigorous verification and validation, and readiness for the emerging exascale computing. Through collaborations with computational scientists, the project helps to advance high performance computing in the area of GPU acceleration, linear solver, and data management.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Developing And Scaling an OpenFOAM Model to Study Turbulent Flow in a HFIR Coolant Channel

Improving the understanding of how computational fluid dynamics (CFD) direct numerical simulations (DNS) of flows in the High Flux Isotope Reactor (HFIR) perform when run in parallel using the high performance computing (HPC) platform Summit at the Oak Ridge Leadership Computing Facility (OLCF) is of particular importance to boost the computational tools used to support HFIR conversion to low enriched fuel (LEU). Evaluation of scaling performance was driven by the increasing importance of graphics processing unit (GPU) usage in HPC, which is becoming the standard for modern supercomputers such as Summit. The desired results are to obtain a strong positive correlation between the computational resources dedicated to a problem and the relative speed-up of the simulation in comparison to a benchmark. This capability will allow substantially improvement in HFIR flow analytical capabilities, specifically when predicting turbulence properties at high Reynolds numbers. The study leverages previous simulation results performed with code PHASTA (finite element) on HPC platforms Cori (NERSC) and Theta (ALCF) [1] with computing options provided in the computing platform OpenFOAM (finite volume) at OLCF. Transitioning from PHASTA to OpenFOAM will (1) eliminate dependence on third-party software for mesh generation and manipulation, (2) reduce resource needs by employing modern architectures, and (3) build expertise for future modeling of HFIR-specific problems like heat transfer in involute geometry, entrance effects, flow structure in channel corners, and so on—all important issues when defining the available thermal margins in the transition to LEU. CPUs and GPUs differ significantly in their architecture and utilization, as discussed in the literature [2]. The most important differences are in the approach to computations and their memory. A single GPU contains a large quantity of cores, enabling it to perform with a much higher throughput than a CPU, but execution requires a different approach. GPU codes execute instructions using the Single-Instruction Multiple-Thread (SIMT) approach in which a single instruction is used for groups of threads called warps. A warp typically consists of 32 threads which must execute the same set of instructions, although on separate threads. Alternately, a CPU has far fewer cores that are much more flexible in their operation, excelling at quickly performing more complex serial computations. This is why GPUs have greater throughput when properly utilized. The second important difference is seen when comparing their memory spaces. Limited memory allocations and CPU–GPU communications cause a significant bottleneck in GPU-accelerated programs. Further study was required to properly take advantage of GPU resources. A comprehensive analysis of code performance and the model-specific features of turbulence constitutes the core of this work. In this study, a DNS simulation of HFIR channel turbulence was performed with the finite volume CFD code OpenFOAM v2112 and CUDA v11.0 on Red Hat Enterprise Linux v8.2. The OpenFOAM installation had AMGx integrated to enable GPU acceleration and utilizes the PETSc4FOAM library. The computational resources and the problem size were scaled on CPU and CPU + GPU architectures to gain a better understanding of the performance of a DNS problem on modern computing hardware. The study aimed to analyze the scaling of the code exclusively on CPUs and then to examine the scaling of the codes with GPU acceleration enabled. Scaling studies included CPU and GPU acceleration on a mesh of varying resolution to analyze the impact of problem size relative to computational resources. In the course of preparing the GPU configuration on Summit, mainly using the AMGX solvers, difficulties were encountered stemming from constant changes resulting from extensive ongoing development activities and the changing environment. This resulted in the inability to complete the GPU portion of the work. The code was compiled and tested, but production runs to assess acceleration were not performed because the used discretional compute time allocation expired as year-end approached. The Summit HPC platform is scheduled for decommissioning in 2024, making it unattractive for future use with Nvidia-based GPUs. Therefore, the work will be moved onto NERSC machines in FY24. An application was prepared and submitted, and sufficient node-hours were awarded to continue the research in the next calendar year. This report summarizes work performed thus far, which mostly focused on CPU OpenFOAM computing.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗