SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
A processor emulator and a suite of benchmark applications have been developed to assist in characterizing the performance of data-centric workloads on current and future computer architectures. Some of the applications have been collected from other open source projects. For more details on the emulator and an example of its usage, see reference [1].
A processor emulator and a suite of benchmark applications have been developed to assist in characterizing the performance of data-centric workloads on current and future computer architectures. Some of the applications have been collected from other open source projects. For more details on the emulator and an example of its usage, see reference [1].
This work presents an assessment of the applicability of existing benchmark critical experiments to the criticality safety code validation for a large-capacity high-assay low-enriched uranium (HALEU) transportation package concept. Numerous next-generation nuclear reactor designs require HALEU fuel, which is characterized by an enrichment between 5 and 20 wt% 235 U. The U.S. Department of Energy (DOE) has proposed to recover and downblend highly enriched uranium from DOE-owned used nuclear fuel to accelerate the demonstration of commercially viable microreactor technologies. One element of the infrastructure needed to demonstrate HALEU-fueled reactors is the ability to safely transport enriched product to be used for fuel fabrication. There is uncertainty as to whether existing critical benchmark experiment data are sufficient to support criticality safety code validation for HALEU transportation applications. The anticipated chemical form of the HALEU in the proposed transportation concept is UO 2 with 20 wt% 235 U/U. The concept uses a combination of an existing transportation packaging design and a novel basket design, including borated aluminum flux traps. The basket provides space for 18 reusable, stainless steel canisters that contain the HALEU. In 10 CFR 71, normal conditions of transport (NCTs) and hypothetical accident conditions (HACs) are defined for fissile material transportation packages. NCT and HAC KENO-VI models of the transportation package were developed using the Standardized Computer Analyses for Licensing Evaluation (SCALE) 6.2.3 computer code package, and optimum moderation conditions were determined using the SCALE SAMPLER sequence. The SCALE Tools for Sensitivity and Uncertainty Analysis Methodology Implementation (TSUNAMI) sequences were then used to compare the neutronic characteristics of 1584 International Criticality Safety Benchmark Evaluation Project benchmark critical experiments with the NCT and HAC HALEU transportation models. The TSUNAMI integral correlation coefficient c k was the criterion used to rank neutronic similarity. Thirty-four experiments were identified as similar (c k ≥ 0.9) to the NCT model, and 55 experiments were identified as similar to the HAC model. Hundreds of experiments were also identified as at least marginally similar (c k ≥ 0.8) to both models. The results indicate that additional critical experiments are unlikely to be needed to support HALEU transportation criticality safety analyses for package concepts similar to the concept package analyzed.
SABATH provides benchmarking infrastructure for evaluating scientific ML/AI models. It contains support for scientific machine learning surrogates from external repositories. The software dependences are explicitly exposed in the surrogate model definition, which allows the use of advanced optimization, communication, and hardware features.
The CABA project is a test of a new tool, Survey,developed by Trenza to be used not only for benchmarking or profiling programs but also to allow incorporation of the information provided by Survey to be utilized in a CI, continuous integration,tool such as GitLab CI.Survey is foremost a means of assessing code performance in terms of time and operations which for computer programmers is known as benchmarking.
We aim at showcasing the scalability of Gaussian Process (GP). The naive GP implementation scales cubically with data size, which can be prohibitive, so GP has not heretofore been considered suitable for very large-scale problem settings. We take advantage of GPyTorch, a library for scalable GPs built on top of PyTorch that incorporates GPU acceleration. With GPyTorch, one can achieve nearly linear scaling with structured kernel interpolation (SKI) and constant-time predictive covariances computation with LanczOs Variance Estimates (LOVE) while preserving accuracy. We also take advantage of the computational power of ThetaGPU, a supercomputer of Argonne Leadership Computing Facility (ALCF). In addition, we implement a scalable, GPU-ready version of Targeted Adaptive Design (TAD), a GP-based data-driven algorithm that efficiently searches the control space of an advanced manufacturing experiment for settings capable of producing a required design within a specified tolerance, despite the poorly known mapping from control settings to design. We finally show our benchmarking for GPyTorch and TAD performance on CPU vs. ThetaGPU and discuss the results and implications.
In a nonlocal game, two noncommunicating players cooperate to convince a referee that they possess a strategy that does not violate the rules of the game. Quantum strategies allow players to optimally win some games by performing joint measurements on a shared entangled state, but computing these strategies can be challenging. We present a variational quantum algorithm to compute quantum strategies for nonlocal games by encoding the rules of a nonlocal game into a Hamiltonian. We show how this algorithm can generate a short-depth optimal quantum strategy for a graph coloring game with a quantum advantage. This quantum strategy is then evaluated on fourteen different quantum hardware platforms to demonstrate its utility as a benchmark. Finally, we discuss potential sources of errors that can explain the observed decreased performance of the executed task and derive an expression for the number of samples required to accurately estimate the win rate in the presence of noise.
Steel Thread is a NA-22 venture that seeks to build trustworthy, reliable AI models that can be used in a wide variety of nonproliferation tasks. A key aspect of building these models is developing appropriate benchmarks and evaluation methods, which will enable the venture to identify and adapt models to provide the most value in the nonproliferation domain. Benchmarks must be relevant to key tasks in this domain, such as question answering, information retrieval, document summarization and classification, consensus analysis, and image and data analysis. This report 1) provides an overview of benchmark design, evaluation, and challenges; 2) reviews a variety of open benchmarks, with a focus on language models and tasks; and 3) identifies benchmarks that are most relevant to Steel Thread. This report is intended to serve as a basis for further efforts to classify and evaluate benchmarks and their correlation with success on nonproliferation-specific tasks. The Steel Thread venture has defined benchmarks to be a particular combination of a dataset (or datasets) and a metric (or metrics) conceptualized as representing one or more specific tasks or sets of abilities for a specific modality. It is adopted by a research community as a shared framework for comparing methods.1 It includes 1) Data: Labeled (a designated subset not used for training, which could be all the data), 2) Metric: A way to quantify performance, 3) Task/Ability: What the benchmark is testing, 4) Protocol: A structured and repeatable evaluation process, 5) Baseline/Reference Model: For comparison; could be statistical, rule-based, SME-derived, or another model, and 6) Maintenance Plan: to update with new information over time; important for long-term utility. For further clarity, the definition includes what a benchmark, in this context, is not. It is not a corpus of training data, specific to a model (it is intended to apply to a range of models), a universal evaluation of performance, a guarantee that the ‘top’ model on the leaderboard will be the best fit for every specific use case, an all-encompassing proof of a model’s universal quality, nor is it a one-size-fits-all measure of success. It does not cover every real-world constraint (like operational, ethical, or cost considerations), a systems integration test, or a unit test. This definition was inspired by and resulted from discussions within the Steel Thread Benchmarking Task Force. This group was formed to define what we would mean as a benchmark within Steel Thread but persisted as the need to develop a thorough understanding of the large and expanding existing benchmarking space. This technical report is a result of the group’s divide and conquer approach to exploring this space. The release of benchmarks might not be progressing as quickly as model development, but it is moving very fast, as many benchmarks quickly become saturated, when state-of-the-art models score so close to the benchmark’s ceiling that their results are virtually indistinguishable. At that point, the test no longer differentiates between new systems, so researchers usually stop reporting scores as the benchmark no longer informs about improvements from the next generation of models. In the OpenAI announcement of GPT-5, they reported results on six flagship public benchmarks (AIME 2025, SWE-bench Verified, Aider Polyglot, MMMU, HealthBench Hard, GPQA) but the full system-card covers roughly thirty-five separate evaluations, comprising hundreds of test task items in total. There have been some efforts to summarize benchmarks in specific fields, like for text-to-image generation, but these surveys have had a narrow methodology scope. Therefore, a comprehensive survey of all benchmarks or even all benchmarks that could be relevant to Steel Thread is outside of the scope of this report. We chose some specific benchmarks to investigate in detail.
The OpenCL standard is an open programming model for accelerating algorithms on heterogeneous computing system. OpenCL extends the C-based programming language for developing portable codes on different platforms such as CPU, Graphics processing units (GPUs), Digital Signal Processors (DSPs) and Field Programmable Gate Arrays (FPGAs). The Intel FPGA SDK for OpenCL is a suite of tools that allows developers to abstract away the complex FPGA-based development flow for a high-level software development flow. Users can focus on the design of hardware-accelerated kernel functions in OpenCL and then direct the tools to generate the low-level FPGA implementations. The approach makes the FPGA-based development more accessible to software users as the needs for hybrid computing using CPUs and FPGAs are increasing. It can also significantly reduce the hardware development time as users can evaluate different ideas with high-level language without deep FPGA domain knowledge. Benchmarking of OpenCL-based framework is an effective way for analyzing the performance of system by studying the execution of the benchmark applications. CHO is a suite of benchmark applications that provides support for OpenCL [1]. The authors presented CHO as an OpenCL port of the CHStone benchmark. Using Altera OpenCL (AOCL) compiler to synthesize the benchmark applications, they listed the resource usage and performance of each kernel that can be successfully synthesized by the compiler. In this report, we evaluate the resource usage and performance of the CHO benchmark applications using the Intel FPGA SDK for OpenCL and Nallatech 385A FPGA board that features an Arria 10 FPGA device. The focus of the report is to have a better understanding of the resource usage and performance of the kernel implementations using Arria-10 FPGA devices compared to Stratix-5 FPGA devices. In addition, we also gain knowledge about the limitations of the current compiler when it fails to synthesize a benchmark application.
Parallel application benchmarks are indispensable for evaluating/optimizing HPC software and hardware. However, it is very challenging and costly to obtain high-fidelity benchmarks reflecting the scale and complexity of state-of-the-art parallel applications. Hand-extracted synthetic benchmarks are time-and labor-intensive to create. Real applications themselves, while offering most accurate performance evaluation, are expensive to compile, port, reconfigure, and often plainly inaccessible due to security or ownership concerns. This work contributes APPRIME, a novel tool for trace-based automatic parallel benchmark generation. Taking as input standard communication-I/O traces of an application's execution, it couples accurate automatic phase identification with statistical regeneration of event parameters to create compact, portable, and to some degree reconfigurable parallel application benchmarks. Experiments with four NAS Parallel Benchmarks (NPB) and three real scientific simulation codes confirm the fidelity of APPRIME benchmarks. They retain the original applications' performance characteristics, in particular the relative performance across platforms.
Neutron transport methods used to establish subcriticality require validation by comparison to critical experiments considered to be benchmarks. Whisper is a sensitivity/uncertainty analysis tool developed to assist with the task of validation in nuclear criticality safety. Details on the Whisper methodology can be found in References 1-3 on the MCNP ® reference collection website at https://mcnp.lanl.gov. Whisper-1.0 was originally developed in 2014 and used to assist with nuclear criticality safety validation at Los Alamos National Laboratory. Whisper was upgraded in 2016 to Whisper-1.1 and prepared for release with MCNP6.2. Whisper contains a library of over 1100 critical experiment benchmarks and quantifies neutronic similarity of an application to benchmarks in the library. Using highest similarity benchmarks, Whisper computes a calculational margin (CM) encompassing of the worst-case bias and bias uncertainty at a 99% confidence level for each application. In addition, portions of the margin of subcriticality (MOS) for nuclear data uncertainty and potential code errors are computed. The baseline upper subcritical limit (USL) computed by Whisper is comprised of the CM, MOS nuclear data , and MOS code errors . The Whisper baseline USL is absent a portion of the MOS due to the area of application, which is applied based upon judgment by the criticality safety analyst. An objective of this paper is to present the baseline USL, CM and portions of the MOS as computed by Whisper for comparison with similar sensitivity/uncertainty tools, such as those used by IRSN and ORNL. The initial comparison involves four critical experiment benchmarks: HEU-MET-FAST-013-001, HEU-SOLTHERM-001-008, PU-MET-FAST-022-001, AND PU-SOL-THERM-001-001, which have been: 1. modeled independently by LANL, IRSN, and ORNL based upon information provided in the ICSBEP Handbook, 2. are common in S/U libraries for LANL, IRSN, and ORNL, 3. span a range of energy spectrum and fissile material, and 4. taken as applications for the purposes of this study and therefore excluded from use as a benchmark for calculating the upper subcritical limit. Results presented in this paper have been computed using covariance data for all isotopes in ENDF/BVII.0 using a 44-group energy structure. Benchmarks in the Whisper library were run in MCNP6.2 using 100,000 neutrons per cycle, skipping 100 cycles for 500 active cycles. Reference 10 also compares the results for baseline USL with an order of magnitude greater neutrons, using the same total number of cycles with 1,000,000 neutrons per cycle. Subsequent to the results presented in Reference 10 changes were made to the benchmark library, as discussed in Reference 11. Newer results using the revised benchmark library are presented herein.
This environmental calculation files (ECF) presents literature benchmarking simulations for the Electrolyte option of the eSTOMP-W simulator and application tot he 216-B-14 salt-water releases in the vadose zone at the BC Cribs area in the 200 East Area of the Hanford Site Central Plateau.
Measurements of gamma-ray emissions from the decay of cosmogenic radionuclides provide an opportunity to characterize the elemental composition of a terrestrial planet or asteroid surface. We report on the development of a Geant4 application which models cosmogenic radionuclide production on metal-rich surfaces. The model was benchmarked using measurements of radionuclides produced during 1 GeV proton irradiation of a target made from the Campo del Cielo iron meteorite. The gamma-ray emitting radionuclides 58 Co, 57 Co, 56 Co, 54 Mn, 52 Mn, 51 Cr, 48 V, 46 Sc and 22 Na were observed following the irradiation. Our model reproduced the measured radionuclide production rates to within a factor of 2.5 or better. All but two of the elements ( 54 Mn, 46 Sc) have a perfect model-to-data match within their measurement uncertainties. The benchmarked model was used to predict cosmogenic radionuclide production on a large (~100-km radius) metal-rich asteroid. The results are relevant for planned gamma-ray measurements of the metallic asteroid (16) Psyche, which will be visited by the Psyche spacecraft in 2026. We found that galactic-proton-induced radionuclide decay is unlikely to be observed by the Psyche Gamma-Ray Spectrometer (GRS), however an intense solar proton event (>2×10 6 protons cm -2 over <5 days) will produce measurable quantities of radioisotopes. Measurements of these radioisotopes could provide an independent method of determining the Ni-to-Fe ratio of materials at Psyche’s surface. Such an analysis will require the use of radionuclide production cross sections to convert GRS-measured cosmogenic radionuclide decay rates to elemental composition information with the ~10% precision required for planetary geochemical studies.
The increasing on-chip parallelism has some substantial implications for HPC applications. Currently, hybrid programming models (typically MPI+OpenMP) are employed for mapping software to the hardware in order to leverage the hardware?s architectural features. In this paper, we present an approach that automatically introduces thread level parallelism into Chombo, a parallel adaptive mesh refinement framework for finite difference type PDE solvers. In Chombo, core algorithms are specified in the ChomboFortran, a macro language extension to F77 that is part of the Chombo framework. This domain-specific language forms an already used target language for an automatic migration of the large number of existing algorithms into a hybrid MPI+OpenMP implementation. It also provides access to the auto-tuning methodology that enables tuning certain aspects of an algorithm to hardware characteristics. Performance measurements are presented for a few of the most relevant kernels with respect to a specific application benchmark using this technique as well as benchmark results for the entire application. The kernel benchmarks show that, using auto-tuning, up to a factor of 11 in performance was gained with 4 threads with respect to the serial reference implementation.
A new immersed boundary-lattice Boltzmann method (IB-LBM) is presented for fully resolved simulations of incompressible viscous flows laden with rigid particles. The immersed boundary method (IBM) recently developed by Breugem (2012) [19] is adopted in the present method, development including the retraction technique, the multi-direct forcing method and the direct account of the inertia of the fluid contained within the particles. The present IB-LBM is, however, formulated with further improvement with the implementation of the high-order Runge–Kutta schemes in the coupled fluid–particle interaction. The major challenge to implement high-order Runge–Kutta schemes in the LBM is that the flow information such as density and velocity cannot be directly obtained at a fractional time step from the LBM since the LBM only provides the flow information at an integer time step. This challenge can be, however, overcome as given in the present IB-LBM by extrapolating the flow field around particles from the known flow field at the previous integer time step. The newly calculated fluid–particle interactions from the previous fractional time steps of the current integer time step are also accounted for in the extrapolation. The IB-LBM with high-order Runge–Kutta schemes developed in this study is validated by several benchmark applications. It is demonstrated, for the first time, that the IB-LBM has the capacity to resolve the translational and rotational motion of particles with the second-order accuracy. The optimal retraction distances for spheres and tubes that help the method achieve the second-order accuracy are found to be around 0.30 and −0.47 times of the lattice spacing, respectively. Simulations of the Stokes flow through a simple cubic lattice of rotational spheres indicate that the lift force produced by the Magnus effect can be very significant in view of the magnitude of the drag force when the practical rotating speed of the spheres is encountered. This finding may lead to more comprehensive studies of the effect of the particle rotation on fluid–solid drag laws. It is also demonstrated that, when the third-order or the fourth-order Runge–Kutta scheme is used, the numerical stability of the present IB-LBM is better than that of all methods in the literature, including the previous IB-LBMs and also the methods with the combination of the IBM and the traditional incompressible Navier–Stokes solver. - Highlights: • The IBM is embedded in the LBM using Runge–Kutta time schemes. • The effectiveness of the present IB-LBM is validated by benchmark applications. • For the first time, the IB-LBM achieves the second-order accuracy. • The numerical stability of the present IB-LBM is better than previous methods.
Not Available
The Jastrow-modified antisymmetric geminal power (JAGP) ansatz in Hilbert space successfully overcomes two key failings of other pairing theories, namely, a lack of inter-pair correlations and a lack of multiple resonance structures, while maintaining a polynomially scaling cost, variational energies, and size consistency. Here, we present efficient quantum Monte Carlo algorithms that evaluate and optimize the JAGP energy for a cost that scales as the fifth power of the system size. We demonstrate the JAGP’s ability to describe both static and dynamic correlation by applying it to bond stretching in H{sub 2}O, C{sub 2}, and N{sub 2} as well as to a novel, multi-reference transition state of ethene. JAGP’s accuracy in these systems outperforms even the most sophisticated single-reference methods and approaches that of exponentially scaling active space methods.