Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Verifying MCNP Models of the TEX High 240 Plutonium Benchmark

Computational modeling programs are invaluable tools that allow us to understand systems, safely develop new processes, and make reliable predictions about future designs. However, the effectiveness of these codes is limited by the degree to which their parameters match the real world. In the field of nuclear engineering, cross section data is one of these vital parameters. Accurate cross section data on important fissile and fissionable isotopes promotes the design of safer and more efficient fabrication, transportation, storage, and stockpiling of nuclear fuel. Unfortunately, there are knowledge gaps in data on key isotopes. In 2011, a multinational meeting hosted by the US Department of Energy Nuclear Criticality Safety Program ranked the priority of certain cross section data needs. In response, Lawrence Livermore National Lab (LLNL) designed the Thermal and Epithermal eXperiment (TEX) series of benchmark experiments. Benchmark experiments are used to validate current cross section data. They validate data by comparing the results of an actual experiment to the predicted results from a computational model. The data a benchmark applies to depends on the isotope and energy range the experiment’s neutron multiplication factor ( k eff ) is most sensitive to. The development and testing of the TEX High 240 Plutonium Benchmark will help validate 240 Pu cross section data. The configuration and materials of this benchmark are designed to be most sensitive to 240 Pu's intermediate energy range (from 0.625 ev to 100 keV ). MCNP® models of the assembly have been developed by LLNL and the results have been written in the final design report. In order for the discrepancies between benchmark models and experiments to be attributed to cross section inaccuracies, the accuracy of the models needs to be verified. The goal of this project is to verify of the results of LLNL's modeling by creating a new set of MCNP models and comparing the results.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Benchmarking NPP Decommissioning Costs - Outcomes of the OECD Nuclear Energy Agency Activity - 19117

This paper describes work at the OECD Nuclear Energy Agency (NEA) on benchmarking in the context of nuclear power plant decommissioning costs. The NEA's Decommissioning Cost Estimation Group (DCEG) benchmarking activity was conducted during 2017-2018, and the report is scheduled for publication by NEA during the first half of 2019. The DCEG's activity focuses on identifying possible benchmarking approaches and discussing their specific application to decommissioning costing, including the 'added value' in developing decommissioning cost benchmarking approaches. It includes consideration of prevailing barriers to the sharing of information and data required for decommissioning cost benchmarking, and explores a range of possible initiatives intended to facilitate implementation of benchmarking approaches in nuclear decommissioning costing (i.e. a possible 'road map' for implementation). (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Revisiting the TORT Solutions to the NEA Suite of Benchmarks for 3D Transport Methods and Codes Over a Range in Parameter Space

Improved TORT solutions to the 3D transport codes' suite of benchmarks exercise are presented in this study. Preliminary TORT solutions to this benchmark indicate that the majority of benchmark quantities for most benchmark cases are computed with good accuracy, and that accuracy improves with model refinement. However, TORT fails to compute accurate results for some benchmark cases with aspect ratios drastically different from 1, possibly due to ray effects. In this work, we employ the standard approach of splitting the solution to the transport equation into an uncollided flux and a fully collided flux via the code sequence GRTUNCL3D and TORT to mitigate ray effects. The results of this code sequence presented in this paper show that the accuracy of most benchmark cases improved substantially. Furthermore, the iterative convergence problems reported for the preliminary TORT solutions have been resolved by bringing the computational cells' aspect ratio closer to unity and, more importantly, by using 64-bit arithmetic precision in the calculation sequence. Results of this study are also reported.

97 MATHEMATICS AND COMPUTING↗

Advanced Benchmarking for Complex Building Types: Laboratories as an Exemplar

Complex buildings such as laboratories, data centers and cleanrooms present particular challenges for energy benchmarking because it is difficult to normalize special requirements such as health and safety in laboratories and reliability (i.e., system redundancy to maintain uptime) in data centers which significantly impact energy use. For example, air change requirements vary widely based on the type of work being performed in each laboratory space. We present methods and tools for energy benchmarking in laboratories, as an exemplar of a complex building type. First, we address whole building energy metrics and normalization parameters. We present empirical methods based on simple data filtering as well as multivariate regression analysis on the Labs21 database. The regression analysis showed lab type, lab-area ratio and occupancy hours to be significant variables. Yet the dataset did not allow analysis of factors such as plug loads and air change rates, both of which are critical to lab energy use. The simulation-based method uses an EnergyPlus model to generate a benchmark energy intensity normalized for a wider range of parameters. We suggest that both these methods have complementary strengths and limitations. Second, we present"action-oriented" benchmarking, which extends whole-building benchmarking by utilizing system-level features and metrics such as airflow W/cfm to quickly identify a list of potential efficiency actions which can then be used as the basis for a more detailed audit. While action-oriented benchmarking is not an"audit in a box" and is not intended to provide the same degree of accuracy afforded by an energy audit, we demonstrate how it can be used to focus and prioritize audit activity and track performance at the system level. We conclude with key principles that are more broadly applicable to other complex building types.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Simplified two and three dimensional HTTR benchmark problems

To assess the accuracy of diffusion or transport methods for reactor calculations, it is desirable to create heterogeneous benchmark problems that are typical of whole core configurations. In this paper we have created two and three dimensional numerical benchmark problems typical of high temperature gas cooled prismatic cores. Additionally, a single cell and single block benchmark problems are also included. These problems were derived from the HTTR start-up experiment. Since the primary utility of the benchmark problems is in code-to-code verification, minor details regarding geometry and material specification of the original experiment have been simplified while retaining the heterogeneity and the major physics properties of the core from a neutronics viewpoint. A six-group material (macroscopic) cross section library has been generated for the benchmark problems using the lattice depletion code HELIOS. Using this library, Monte Carlo solutions are presented for three configurations (all-rods-in, partially-controlled and all-rods-out) for both the 2D and 3D problems. These solutions include the core eigenvalues, the block (assembly) averaged fission densities, local peaking factors, the absorption densities in the burnable poison and control rods, and pin fission density distribution for selected blocks. Also included are the solutions for the single cell and single block problems.

99 GENERAL AND MISCELLANEOUS↗

Oak Ridge OpenSHMEM Benchmark Suite

The assessment of application performance is a fundamental task in high-performance computing (HPC). The OpenSHMEM Benchmark (OSB) suite is a collection of micro-benchmarks and mini-applications/compute kernels that have been ported to use OpenSHMEM. Some, like the NPB OpenSHMEM benchmarks, have been published before while most others have been used for evaluations but never formally introduced or discussed. This suite puts them together and is useful for assessing the performance of different use cases of OpenSHMEM. This offers system implementers a useful means of measuring performance and assessing the effects of new features as well as implementation strategies. The suite is also useful for application developers to assess the performance of the growing number of OpenSHMEM implementations that are emerging. In this paper, we describe the current set of codes available within the OSB suite, how they are intended to be used, and, where possible, a snapshot of their behavior on one of the OpenSHMEM implementations available to us. We also include detailed descriptions of every benchmark and kernel, focusing on how OpenSHMEM was used. This includes details on the enhancements we made to the benchmarks to support multithreaded variants. We encourage the OpenSHMEM community to use, review, and provide feedback on the benchmarks.

Naughton III, Thomas↗

Depletion benchmark for a high-assay low-enriched uranium fuel experiment in the advanced test reactor

Reactor physics depletion benchmarks for high-assay low-enriched uranium (HALEU) fuel are limited in number. In particular, there is limited data for HALEU benchmarks for U-10Mo (uranium-10% molybdenum) plate fuel that is being developed for use in the United States’ high performance research reactors including the Advanced Test Reactor (ATR), Advanced Test Reactor Critical Facility (ATR-C), High Flux Isotope Reactor (HFIR), Massachusetts Institute of Technology Reactor (MITR), University of Missouri Research Reactor (MURR), National Bureau of Standards Reactor (NBSR). These six reactors currently operate with highly enriched uranium dispersed fuel in an aluminum matrix. In support of conversion to a HALEU fuel, qualification of U-10Mo formed into a monolithic foil is being performed. Fuel qualification involves irradiating fuel specimens in the ATR. The irradiation tests provide an opportunity to benchmark depletion capabilities of reactor physics codes in support of the ATR operation, as well as develop benchmarks that can be used by other institutions to benchmark other reactor physics codes. This paper documents the development of a benchmark model of the irradiation of the ATR Full-size plate In center flux trap Position 7 (AFIP-7) experiment using the depletion codes MC21 and Advanced Dimensional Depletion for Engineering of Reactors (ADDER).

Nielsen, Joseph W. [Idaho National Laboratory (INL↗

On benchmarking of simulations of particle transport in ITER

We present results of benchmarking of core particle transport simulations by a collection of codes widely used in transport modelling of tokamak plasmas. Our analysis includes formulation of transport equations, difference between electron and ion solvers, comparison of modules of the pellet and edge gas fuelling on the ITER baseline scenario. During the first phase of benchmarking we address the particle transport effects in the stationary phase. Firstly, simulations are performed with identical sources, sinks, transport coefficients, and boundary conditions prescribed in the flattop H-mode phase. The transformation of ion particle transport equations is introduced so to directly compare their results to electron transport solvers. Moreover, the pellet fuelling models are benchmarked in various conditions to evaluate the dependency of the pellet deposition on the pellet volume, injection side, pedestal and separatrix parameters. Thirdly, edge gas fuelling is benchmarked to assess sensitivities of source profile predictions to uncertainties in plasma conditions and detailed model assumptions. At the second phase, we address particle transport effects in the time-evolving plasma including the current ramp-up to the ramp-down phase. The ion and the electron solvers are benchmarked together. Differences between the simulation results of the solvers are investigated in terms of equilibrium, grid resolution, radial coordinate, radial grid distribution, and plasma volume evolution term. We found that the selection of the radial coordinate can yield prominent differences between the solvers mainly due to differences in the edge grid distribution. The simulations reveal that electron and ion solvers predict noticeably different density peaking for the same diffusion and pinch velocity while with the peaked profile of helium, expected in fusion reactors. The fuelling benchmarking demonstrates that gas puffing is not efficient for core fuelling in H-modes and density control should be done by the high field side pellet injection in contrast to present machines.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

BuildingQA: A Benchmark for Natural Language Question Answering over Building Knowledge Graphs

Graph-based representations of building metadata using ontologies like Brick are vital for smart building applications, but querying them remains a challenge for practitioners. Knowledge Graph Question Answering (KGQA) systems, meant to retrieve answers from natural language questions, traditionally require large-scale training data, making them ill-suited for the specialized and data-scarce building domain. The advent of Large Language Models (LLMs) offers a paradigm shift, enabling zero-shot natural language querying without building/domain-specific training. Yet, there is no standardized benchmark for building-specific KGQA which can guide and validate research in this area. To address this gap, our work makes three primary contributions. First, we introduce the BuildingQA Benchmark Dataset, constructed through a multi-stage process of collecting practitioner data, augmenting it with LLMs for linguistic diversity, and curating a final set of 188 questions across 4 buildings. Second, we characterize the benchmark's complexity and ambiguity, introducing a novel method to quantify its "lexical gap" and providing a four-stage diagnostic framework for analyzing how systems fail. Third, we benchmark zero-shot LLM-powered KGQA systems to establish baseline performance and analyze their failure modes. Our evaluation reveals that top-performing systems achieve a maximum F1 score of only 0.38. This result does not indicate a failure of these powerful systems, but rather underscores the unique challenges posed by our benchmark. It demonstrates a critical performance gap, showing that current methods successful on general KGs struggle with the specific lexical and structural nuances of the building domain. BuildingQA1 thus provides the benchmark dataset and foundational analysis needed to drive the development of novel, domain-aware methods required to unlock the use of semantic data in buildings.

Mulayim, Ozan Baris↗

Validation of MCNP Critical Benchmark Models of PU-MET-FAST-016

A new centralized repository of high-quality MCNP models of critical benchmark experiments is currently under development at Los Alamos National Laboratory (LANL). The repository is the combined effort of the Nuclear Criticality Safety, Nuclear Data, and Monte Carlo code development/application organizations at LANL. The initial set of benchmark models in the repository are derived from the Whisper Suite provided with MCNP6.2 and are evaluated against the benchmark experiments described in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook. The goal is to build a single LANL benchmark collection that is up to date with the latest ICSBEP revision, has a formal review and revision process, is contained in an open-source repository, and utilizes new Python tools for improved input and output file review. This paper describes the validation of the models associated with PU-MET-FAST-016, “Flooded 3X3X3 Arrays of 3-kg Plutonium (Pu) Metal Cylinders – Phase I”. In this critical experiment, twenty-seven 3-kg Pu metal cylinders with a height of 46.33 ± 0.15 mm and diameter of 65.25 ± 0.05 mm were arranged in a cubic array. The Pu cylinders were sealed in aluminum cans, and three cans were placed in each of nine perforated aluminum sleeves, as shown in Fig. 1. The nine sleeves were supported by an aluminum frame that allowed for control of the lateral spacing of the cylinders, both in the X and Y directions. The benchmark experiment measured thirteen configurations. Of these configurations, six were modeled using MCNP.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Radiation Detection Computational Benchmark Scenarios

Modeling forms an important component of radiation detection development, allowing for testing of new detector designs, evaluation of existing equipment against a wide variety of potential threat sources, and assessing operation performance of radiation detection systems. This can, however, result in large and complex scenarios which are time consuming to model. A variety of approaches to radiation transport modeling exist with complementary strengths and weaknesses for different problems. This variety of approaches, and the development of promising new tools (such as ORNL’s ADVANTG) which combine benefits of multiple approaches, illustrates the need for a means of evaluating or comparing different techniques for radiation detection problems. This report presents a set of 9 benchmark problems for comparing different types of radiation transport calculations, identifying appropriate tools for classes of problems, and testing and guiding the development of new methods. The benchmarks were drawn primarily from existing or previous calculations with a preference for scenarios which include experimental data, or otherwise have results with a high level of confidence, are non-sensitive, and represent problem sets of interest to NA-22. From a technical perspective, the benchmarks were chosen to span a range of difficulty and to include gamma transport, neutron transport, or both and represent different important physical processes and a range of sensitivity to angular or energy fidelity. Following benchmark identification, existing information about geometry, measurements, and previous calculations were assembled. Monte Carlo results (MCNP decks) were reviewed or created and re-run in order to attain accurate computational times and to verify agreement with experimental data, when present. Benchmark information was then conveyed to ORNL in order to guide testing and development of hybrid calculations. The results of those ADVANTG calculations were then sent to PNNL for compilation. This is a report describing the details of the selected Benchmarks and results from various transport codes.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Excluding Benchmark Statistical Outliers in Nuclear Criticality Safety Validation: A Comparison Study of Upper Subcritical Limits for Plutonium Systems using Whisper-1.1

Neutron transport methods used to establish subcriticality require validation by comparison to critical experiments considered to be benchmarks. Whisper is a sensitivity/uncertainty analysis tool developed to assist with the task of validation in nuclear criticality safety. Details on the Whisper methodology can be found in References 1-3 on the MCNP® reference collection website at https://mcnp.lanl.gov. During the process of validation there can be cases where a benchmark experiment may be found to be a statistical outlier, in which the calculated k-effective value and the experiment k-effective value differ by an amount atypical for similar experiments. A methodology optionally employed by Whisper is the exclusion of statistical outliers based upon the iterative diagonal chi-squared statistical rejection technique. Alternatively, there is an option to include all benchmarks in the Whisper library collection, even those benchmarks found to be statistical outliers, when computing the bias, bias uncertainty and margin of subcriticality (MOS) leading to establishment of the baseline upper subcritical limit (USL). A comparison study has been done to compute USLs with and without statistical outliers in the Whisper benchmark collection to determine what effect rejection of statistical outliers has on the recommended USL. The results show little overall difference in the recommended baseline USLs developed by Whisper when excluding statistical outliers. Additionally, there does not appear to be a clear trend in predicting whether the baseline USL will be higher or lower when rejecting statistical outliers from the benchmark critical experiment collection used for validation.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

LANL Critical Benchmark Comparison Study and Subsequent Revision

As part of an international collaboration within the DOE Nuclear Criticality Safety Program (NCSP), LANL is involved in a comparison study to quantify differences in k-effective results from neutron transport simulations of critical benchmark experiments. The DOE NCSP Mission and Vision details the activity in which the French Institut De Radioprotection et De Sûreté Nucléaire (IRSN) leads the study with LANL and in conjunction with ORNL and LLNL to compare results of various neutron transport codes and nuclear data libraries to compute k-effective for ICSBEP benchmarks held in common by the entities. This report documents results obtained through partial completion of the overall effort with a focus on the changes made to LANL benchmarks modeled with MCNP6 using ENDF/B-VII.1 nuclear data that appeared to have discrepant results when compared with results of other codes. The feedback received through participation in the comparison collaboration has prompted an effort to review particular input files for benchmarks and revise when necessary. This report documents the results of review and revision of specific benchmarks highlighted as possibly discrepant in the comparison study. In addition, this effort prompted a new collaboration between LANL XCP and NCS Divisions in the development of a shared review/revision procedure and use of a new benchmark repository.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

IER-517: Molybdenum Optimized Benchmark System Demonstrating Integral Correlations (MOBY DICK)

Nuclear criticality experiments are essential to the validation of nuclear data used in simulation software. The quality of nuclear data becomes paramount as simulation software becomes more relied upon for criticality safety studies and designs of nuclear systems. To improve the quality of nuclear data, experimenters can design critical experiments that are sensitive to isotope reaction pairs in materials of interest. The efforts conducted by the Organisation for Economic Co-operation and Development - Nuclear Energy Agency (OECD-NEA) Working Party on Nuclear Criticality Safety (WPNCS) Subgroup 8: Preservation of Expert Knowledge and Judgement Applied to Criticality Benchmarks (SG8) to categorize benchmarks according to their usefulness for nuclear data validation have been of great importance. Based on the OECD studies benchmark experiments included in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook are concisely used by nuclear data evaluators, criticality safety engineers and others to validate nuclear data and simulation results. A lack of benchmarks sensitive to molybdenum in the (ICSBEP), particularly in the intermediate range, was noted by Los Alamos National Laboratory (LANL), the French Institut de Radioprotection et de Sûreté Nucléaire (IRSN), and Y-12 National Security Site prompting them to submit a joint integral experiment request to the Nuclear Criticality Safety Program (NCSP) in 2019. The request included both HEU and Plutonium systems in order to validate differential nuclear data focusing on the intermediate energy range but also includes thermal and fast configurations. This document represents the preliminary design work for a series of molybdenum integral experiments known as Molybdenum Optimized Benchmark System Demonstrating Integral Correlations (MOBY DICK).

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Machine Learning meets Algebraic Combinatorics: A Suite of Benchmark Datasets to Accelerate AI for Mathematics Research

The use of benchmark datasets has become an important engine of progress in machine learning (ML) over the past 15 years. Recently there has been growing interest in utilizing machine learning to drive advances in research-level mathematics. However, off-the-shelf solutions often fail to deliver the types of insights required by mathematicians. This suggests the need for new ML methods specifically designed with mathematics in mind. The question then is: what benchmarks should the community use to evaluate these? On the one hand, toy problems such as learning the multiplicative structure of small finite groups have become popular in the mechanistic interpretability community whose perspective on explainability aligns well with the needs of mathematicians. While toy datasets are a useful benchmark for initial work, they lack the scale, complexity, and sophistication of many of the principal objects of study in modern mathematics. To address this, we introduce a new collection of benchmark datasets, Algebraic Combinatorics Benchmarks (ACBench), representing either classic or open problems in algebraic combinatorics, a subfield of mathematics that studies discrete structures arising from abstract algebra. After describing the datasets, we discuss the challenges involved in constructing “good” mathematics benchmarks, describe baseline model performance, and discuss some of the insights these datasets can provide that may be of interest even to those who are not interested in mathematics research itself.

97 MATHEMATICS AND COMPUTING↗

Ranking and Classifying AI Benchmarks

We created a set of standards to efficiently evaluate AI benchmarks through objective means. Although prevalent, especially in recent times, AI benchmarks have no single way to measure their effectiveness. The MLCommons team provided a set of criteria for evaluating benchmarks, although the criteria lacks a clearly defined set of evaluation rules. We created a rubric with preset factors to efficiently and objectively evaluate a benchmark s quality. We created a software framework for processing lists of benchmarks for visualization. The framework and rating system allows researchers to quickly check if their benchmarks are effective.

Shiraishi, Reece C. [Cornell U.]↗

A High-Fidelity Model of the Peach Bottom 2 Turbine-Trip Benchmark Using VERA

This work presents a high-fidelity simulation of the Peach Bottom turbine trip (PBTT) benchmark using the Virtual Environment for Reactor Applications (VERA), a multiphysics reactor modeling tool developed by the U.S. Department of Energy’s Consortium for Advanced Simulation of Light Water Reactors energy innovation hub. The PBTT benchmark, based on a 1977 transient event at the end of cycle 2 in a General Electric Type-4 boiling water reactor (BWR), is a critical test case for validating core physics models with thermal feedback during rapid reactivity events. VERA was employed to perform end-to-end, pin-resolved simulations from conditions at the beginning of cycle 1 through the turbine-trip transient, incorporating detailed neutron transport, fuel depletion, and subchannel thermal hydraulics. The simulation reproduced key benchmark observables with high accuracy: the peak power excursion occurred at 0.75 s, matching the scram time and closely aligning with the benchmark average of 0.742 s; the simulated maximum power spike was approximately 7600 MW, which is within 3% of the benchmark average of 7400 MW; and void-collapse dynamics were consistent with benchmark expectations. Reactivity predictions during cycles 1 and 2 remained within 1500 pcm and 400 pcm of criticality, respectively. These results confirm VERA’s ability to model complex coupled neutronic and thermal hydraulic behavior in a BWR turbine-trip transient, which will support its use in future studies of modeling dryout, fuel performance, and uncertainty quantification for transients of this type.

BWR↗

Analytical benchmark solution for 1-D neutron transport coupled with thermal conduction and material expansion

In this work we present fully analytical solutions for a class of finite, homogeneous, 1-D slab benchmark problems with nonlinear temperature feedback effects. The proposed class of benchmarks include multiplicative 1-D neutron transport (limited to quasistatic S{sub 2} with μ = ±1) coupled with thermal conduction, convection, Doppler broadening, and expansion effects along the length of the slab. This class of benchmark models, along with the corresponding analytical solutions, are valuable for validating multiphysics analysis frameworks that support coupled neutronics/thermal/structural calculations. Analytical solutions for the benchmarks are obtained by introducing an ansatz that the equilibrium flux and temperature distributions in the slab have the same shape. Specific values for the total microscopic cross section, σ{sub t,0}, and conductive heat transfer coefficient, h, that satisfy the assumed ansatz are then determined. A discussion of the procedure for generating benchmark models and analytical solutions is provided, along with numerical results for an example set of model parameters and thoughts on practical applications of the benchmarks for multiphysics code validation. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗