Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

Evaluation of Oak Ridge National Laboratory Health Physics Research Reactor Operation Data for Critical Benchmark Creation [Slides]

During the evaluation of Oak Ridge National Laboratory Health Physics Research Reactor operation data for critical benchmark creation, a detailed, functional SCALE model was built, and experimental data was evaluated for the creation of a critical benchmark. The estimated experimental uncertainty is about 3800 pcm, which is considered very high. The relative difference between expected and calculated keff values is about 1.5 %, also considered very high. It was concluded that a good quality critical benchmark worthy of the ICSBEP standards cannot be created from HPRR data in the present conditions. Locating the HPRR fuel to obtain an uncertainty on the density would aid in solving this issue.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Benchmark Specifications for TREAT Tests M5, M6, and M7

Detailed information describing three TREAT tests performed on metallic fuels has been collected and organized for use as a benchmark in evaluating the performance of metallic fuel models and codes. The tests, designated M5, M6, and M7, subjected EBR-II-irradiated fuel pins to a single type of overpower transient (at prototypical conditions with full coolant flow and an exponential power rise on an 8 second period) in flowing sodium loops. Six fuel pins were tested; five were ternary (U-19Pu-10Zr) alloy fuel clad in D9 with burnup ranging from 0.8 to 9.8 at. %, and one was binary alloy fuel (U-10Zr) clad in HT9 with 2.9 at. % burnup. The information gathered from the test records is expected to be useful for pre-transient characterization of the irradiated fuel pins as well as the transient analysis of the metallic fuel when subjected to severe accident conditions. This report presents benchmark specifications for the M5, M6, and M7 TREAT tests, identifies where primary sources of benchmark-related information can be found, and includes background information to help a user of the data understand their applicability and limitations.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

ICSBEP Benchmark Template

The purpose of this document is to demonstrate the ability of LATEX to create an ICSBEP benchmark evaluation. Creating a large document in LATEX offers many benefits, such as automatic numbering of items such as tables and figures, and easier placement of objects in the page. Additionally, it is easier to have multiple styles of pages in the same document, and in general the handling of large documents (which evaluations very commonly are) is not problematic. The formatting of this document is meant to conform to both the standards of the document and formatting guide and established practices in order to make the creation of new evaluations a more streamlined process. This will also help reduce the benchmark-to-benchmark variation in style.

97 MATHEMATICS AND COMPUTING↗

Shielding Benchmark Comparison - MCNP6.2

This report documents the calculations performed for a shielding code comparison between various Department of Energy (DOE) sites. These shielding calculations were performed using an experiment drawn from the International Handbook of Evaluated Criticality Safety Benchmark Experiments, published by the Organisation for Economic Cooperation and Development/Nuclear Energy Agency (OECD/NEA). The benchmark selected for comparison is ALARM-CF-AIR-LAB-001 (“Neutron Fields in the Three-Section Concrete Labyrinth from Cf-252 Source, Benchmark ALARM CF AIR LAB-001”). The Y-12 submission for this code comparison was performed with Monte Carlo N Particle (MCNP) Transport Code System, Version 6.2 and Automated Variance Reduction Generator (ADVANTG).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

wa-hls4ml and lui-gnn: A benchmark and GNN-based surrogate model for hls4ml resource and latency estimation

As machine learning (ML) increasingly serves as a tool for addressing real-time challenges in scientific applications, the development of advanced tooling has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as model synthesis, are now becoming limiting factors in the rapid iteration of designs. To reduce these emerging constraints, multiple efforts are being launched toward designing an ML-based surrogate model that estimates resource usage of synthesized accelerator architectures. This model would reduce the design iteration time, especially when designing within a set of given hardware constraints. This approach shows considerable potential, but as it stands, the effort is early and would benefit from coordination and standardization to assist future work as it emerges. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of more than 100,000 fully connected neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. In addition to the resource utilization and latency data provided, the dataset includes generated artifacts and log files for many of the synthesized neural networks, in order to support future research in ML-based code generation. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, as well as the average performance across a subset of the dataset. We measure the performance of a given predictor model through multiple metrics, including $R^2$ score and SMAPE on regression tasks, as well as inference time to further characterize the estimator under test. Additionally, we introduce the latency/utilization inference graph neural network (lui-gnn), a surrogate model that uses a graph neural network to represent input architectures in the form of a directed graph. This graph representation allows for a diverse set of model architectures to all be effectively handled by a surrogate model. We present the architecture and performance of the model, as evaluated by the new proposed benchmark, including SMAPE, $R^2$ score, and inference times, and find that lui-gnn generally predicts latency and utilization for the 75\% quantile within several percent of the synthesized resources on the synthetic test dataset, indicating that this approach of estimating resource and latency via a surrogate models has promise and warrants further research.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Benchmarking of Probabilistic Fracture Mechanics Models in Grizzly

In 2020, as part of contract/task order 31310019N0006/31310020F0060, the Nuclear Regulatory Commission tasked Idaho National Laboratory (INL) with benchmarking the MOOSE/GRIZZLY code against the NRC’s Fracture Analysis of Vessels - Oak Ridge (FAVOR), version 16.1 code. Both codes can predict large light-water reactor vessel integrity deterministically and probabilistically. In fact, such predictions are the primary purpose of the FAVOR code, and are a main focus of the GRIZZLY code, although GRIZZLY is capable of modeling a wider range of geometries and problems. The vessel integrity modeling capabilities of the GRIZZLY code are still under development and being gradually expanded, and the capability to model vessel fracture by crack growth has not yet been integrated into GRIZZLY (although it does exist in pre-release versions). Thus, the present benchmark study focused on comparing predictions for the conditional probability of crack growth initiation (CPI) between GRIZZLY and FAVOR v16.1. The present benchmarking study expands on a previous study completed in 2020, which focused on single plate regions, various flaw types, and full vessel studies. The previous study was limited in scope and revealed some relatively large discrepancies, particularly in the case of weld regions. In the present study, detailed investigations into the sources of discrepancies between GRIZZLY and FAVOR were performed, with a focus on weld regions.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Using FIPD and OPTD to Benchmark Metallic Fuel Performance

This report serves as an introduction, tutorial, and benchmark specification for out-of-pile tests on metallic fuel. It introduces a new user to the EBR-II legacy fuel performance test program and the fast reactor fuel performance databases built to preserve the records. It then details the information stored in each database and how to find it. A benchmark specification is included for a small set of out-of-pile tests on U-10Zr fuel to function as a tutorial demonstrating how the legacy fuel performance data sets stored in the FIPD and OPTD databases can be used together to benchmark fuel performance models for steady-state and transient performance.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Continuous-Energy Verification of MCNP Calculations Using One-Group Spherical and Slab Criticality Benchmarks

This work presents a continuous-energy Monte Carlo verification study of one group spherical and slab criticality benchmarks using the MCNP ®1 code. Classical tabulations and newly generated benchmark solutions obtained by direct numerical evaluation by the author are considered. The benchmarks span weakly to strongly multiplying regimes and provide analytically defined critical radii as functions of a single parameter, c .

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

The Aerosol Model Benchmarking Repository: A toolkit for model intercomparison

The Aerosol Model Benchmarking Repository and Standards (AMBRS) project was initiated to provide tools and to establish community standards for benchmarking aerosol models. This report describes a set of open-source tools for building, running, and analyzing aerosol box model simulations in a standardized framework. The framework consists of three core components: AMBuilder, a CMake-based build system that compiles supported models consistently; AMBRS, a Python module that defines unified numerical experiments and executes them with aligned inputs; and PyParticle, an aerosol analysis package that standardizes output, computes diagnostics, and visualizes simulation results. Together, these tools enable reproducible intercomparison of aerosol schemes and support process-level evaluation of how model simplifications affect predictions of size distributions, cloud condensation nuclei activity, and other relevant properties relevant for the Earth-Energy system. Beyond its role in benchmarking, AMBRS provides a platform for studying aerosol processes across scales and can be used to generate training data for AI/ML applications in support of a broader hierarchical aerosol modeling strategy.

54 ENVIRONMENTAL SCIENCES↗

Monitoring Based Commissioning: Benchmarking Analysis of 24 UC/CSU/IOU Projects

Buildings rarely perform as intended, resulting in energy use that is higher than anticipated. Building commissioning has emerged as a strategy for remedying this problem in non-residential buildings. Complementing traditional hardware-based energy savings strategies, commissioning is a 'soft' process of verifying performance and design intent and correcting deficiencies. Through an evaluation of a series of field projects, this report explores the efficacy of an emerging refinement of this practice, known as monitoring-based commissioning (MBCx). MBCx can also be thought of as monitoring-enhanced building operation that incorporates three components: (1) Permanent energy information systems (EIS) and diagnostic tools at the whole-building and sub-system level; (2) Retro-commissioning based on the information from these tools and savings accounting emphasizing measurement as opposed to estimation or assumptions; and (3) On-going commissioning to ensure efficient building operations and measurement-based savings accounting. MBCx is thus a measurement-based paradigm which affords improved risk-management by identifying problems and opportunities that are missed with periodic commissioning. The analysis presented in this report is based on in-depth benchmarking of a portfolio of MBCx energy savings for 24 buildings located throughout the University of California and California State University systems. In the course of the analysis, we developed a quality-control/quality-assurance process for gathering and evaluating raw data from project sites and then selected a number of metrics to use for project benchmarking and evaluation, including appropriate normalizations for weather and climate, accounting for variations in central plant performance, and consideration of differences in building types. We performed a cost-benefit analysis of the resulting dataset, and provided comparisons to projects from a larger commissioning 'Meta-analysis' database. A total of 1120 deficiency-intervention combinations were identified in the course of commissioning the projects described in this report. The most common location of deficiencies was in HVAC equipment (65% of sites), followed by air-handling and distributions systems (59%), cooling plant (29%), heating plants (24%), and terminal units (24%). The most common interventions were adjusting setpoints, modifying sequences of operations, calibration, and various mechanical fixes (each done in about two-thirds of the sites). The normalized rate of occurrence of deficiencies and corresponding interventions ranged from about 0.1/100ksf to 10/100ksf, depending on the issue. From these interventions flowed significant and highly cost-effective energy savings For the MBCx cohort, source energy savings of 22 kBTU/sf-year (10%) were achieved, with a range of 2% to 25%. Median electricity savings were 1.9 kWh/sf-year (9%), with a range of 1% to 17%. Peak electrical demand savings were 0.2 W/sf-year (4%), with a range of 3% to 11%. The aggregate commissioning cost for the 24 projects was $2.9 million. We observed a range of normalized costs from $0.37 to 1.62/sf, with a median value of $1.00/sf for buildings that implemented MBCx projects. Per the program design, monitoring costs as a percentage of total costs are significantly higher in MBCx projects (median value 40%) than typical commissioning projects included in the Meta-analysis (median value of 2% in the commissioning database). Half of the projects were in buildings containing complex and energy-intensive laboratory space, with higher associated costs. Median energy cost savings were $0.25/sf-year, for a median simple payback time of 2.5 years. Significant and cost-effective energy savings were thus obtained. The greatest absolute energy savings and shortest payback times were achieved in laboratory-type facilities. While impacts varied from project to project, on a portfolio basis we find MBCx to be a highly cost-effective means of obtaining significant program-level energy savings across a variety of building types. Energy savings are expected to be more robust and persistent for MBCx projects than for conventionally commissioned ones. Impacts of future programs can be maximized by benchmarking energy use and targeting the commissioning towards particularly energy-intensive facilities such as laboratories.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Sustaining knowledge in the neutron generator community and benchmarking study. Phase II.

This report documents the second phase of work under the Sustainable Knowledge Management (SKM) project for the Neutron Generator organization at Sandia National Laboratories. Previous work under this project is documented in SAND2008-1777, Sustaining Knowledge in the Neutron Generator Community and Benchmarking Study. Knowledge management (KM) systems are necessary to preserve critical knowledge within organizations. A successful KM program should focus on people and the process for sharing, capturing, and applying knowledge. The Neutron Generator organization is developing KM systems to ensure knowledge is not lost. A benchmarking study involving site visits to outside industry plus additional resource research was conducted during this phase of the SKM project. The findings presented in this report are recommendations for making an SKM program successful. The recommendations are activities that promote sharing, capturing, and applying knowledge. The benchmarking effort, including the site visits to Toyota and Halliburton, provided valuable information on how the SEA KM team could incorporate a KM solution for not just the neutron generators (NG) community but the entire laboratory. The laboratory needs a KM program that allows members of the workforce to access, share, analyze, manage, and apply knowledge. KM activities, such as communities of practice (COP) and sharing best practices, provide a solution towards creating an enabling environment for KM. As more and more people leave organizations through retirement and job transfer, the need to preserve knowledge is essential. Creating an environment for the effective use of knowledge is vital to achieving the laboratory's mission.

07 ISOTOPE AND RADIATION SOURCES↗

Numerical Benchmark for High-Reynolds-Number Supercritical Flows with Large Density Gradients

Because of the extreme complexity of physical phenomena at high pressure, only limited data are available for solver validation at device-relevant conditions such as liquid rocket engines, gas turbines, or diesel engines. In the present study, a two-dimensional direct numerical simulation is used to establish a benchmark for supercritical flow at a high Reynolds number and high-density ratio at conditions typically encountered in liquid rocket engines. Emphasis has been placed on maintaining the flow characteristics of actual systems with simple boundary conditions, grid spacing, and geometry. Results from two different state-of-the-art codes, with markedly different numerical formalisms, are compared using this benchmark. The strong similarity between the two numerical predictions lends confidence to the physical accuracy of the results. The established database can be used for solver benchmarking and model development at conditions relevant to many propulsion and power systems.

Engineering↗

ILAMBv2.7 benchmarking results comparing E3SMv2.1 land-atmosphere coupled (BGCv2LNDATM) and stand alone land (ELM) simulations with CMIP6 emission driven historical simulations

This dataset contains land model benchmarking results for the Energy Exascale Earth System Model version 2.1 (E3SMv2.1), including outputs from both coupled biogeochemistry simulations and stand-alone land model simulations. These results are compared against several emission-driven historical simulations from the Coupled Model Intercomparison Project Phase 6 (CMIP6). Benchmarking was conducted using the International Land Model Benchmarking (ILAMB) package, version 2.7 (ILAMBv2.7). CMIP6 model outputs were sourced from the Earth System Grid Federation (ESGF), while the E3SMv2.1 results were derived from raw model outputs. These outputs underwent processing steps such as time serialization, conservative regridding, and data standardization to ensure comparability. For spatial interpolation, the Earth System Modeling Framework (ESMF) tool, ESMF_RegridWeightGen, was employed to generate regridding weights, enabling the transformation of E3SM’s native cubed-sphere grid to a regular latitude-longitude grid.

Feng, Sha [PNNL]↗

International Collaboration Framework for the Calculation of Performance Loss Rates: Data Quality, Benchmarks, and Trends

Within the IEA PVPS Task 13, a group of experts representing several leading R&D centres, universities and industry companies, is developing a framework for the calculation of Performance Loss Rates (PLR) on a large number of commercial and research PV power plants and related weather data coming from various climatic zones. Various methodologies are applied for the calculation of PLR which are benchmarked in terms of uncertainties and 'true' values. The aim of the international collaboration is to show how to calculate the PLR on high quality (high time resolution, reliable data, irradiance, yield, etc.) and on low quality data (low time resolution, only energy data available). Various algorithms and models, along with different time averaging and filtering criteria, can be applied for the calculation of PLR each of which can have an impact on the results. The approach considers three pathways to ensure broad collaboration and increase the statistical relevance of the study: i) use of shared methodologies on shared time series, ii) use of confidential methodologies on shared time series, iii) use of shared methodologies on confidential time series. The data is used for benchmarking activities and to define which methodologies clusters around a 'true value' of PLR. The combination of metrics (PR or power based) and methodologies are benchmarked in terms of deviation from the average value and in terms of standard deviation. The combination of temperature corrected PR with the use of Year on Year (YOY) or Seasonal Trend decomposition using Loess (STL) performs well for all systems; other methodologies seems to be more depending on the filtering applied. Another set of data is represented by the IEA PVPS Task 13 'PV performance database' which includes more than 120 PV plants from different climates. The applied methodologies were STL and the YoY approach. STL results in an averaged PLR over all systems of -0.78%/a while YoY yields -0.63%/a. STL is better suited if the time series data are of higher resolution and high-quality weather data are available. One important outcome of this study is the setting of a framework for the contribution to a clear and structured quality classification for PV time series data and a guideline for PLR calculations in dependency of the data categorization.

14 SOLAR ENERGY↗

Nuclear Data Adjustment for Nonlinear Applications in the OECD/NEA WPNCS SG14 Benchmark -- A Bayesian Inverse UQ-based Approach for Data Assimilation

The Organization for Economic Cooperation and Development (OECD) Working Party on Nuclear Criticality Safety (WPNCS) proposed a benchmark exercise to assess the performance of current nuclear data adjustment techniques applied to nonlinear applications and experiments with low correlation to applications. This work introduces Bayesian Inverse Uncertainty Quantification (IUQ) as a method for nuclear data adjustments in this benchmark, and compares IUQ to the more traditional methods of Generalized Linear Least Squares (GLLS) and Monte Carlo Bayes (MOCABA). Posterior predictions from IUQ showed agreement with GLLS and MOCABA for linear applications. When comparing GLLS, MOCABA, and IUQ posterior predictions to computed model responses using adjusted parameters, we observe that GLLS predictions fail to replicate computed response distributions for nonlinear applications, while MOCABA shows near agreement, and IUQ uses computed model responses directly. We also discuss observations on why experiments with low correlation to applications can be informative to nuclear data adjustments and identify some properties useful in selecting experiments for inclusion in nuclear data adjustment. Performance in this benchmark indicates potential for Bayesian IUQ in nuclear data adjustments.

FOS: Computer and information sciences↗

Evaluation of HEU-Beryllium Benchmark Experiments to Improve Computational Analysis of Space Reactors

An assessment was previously performed to evaluate modeling capabilities and quantify preliminary biases and uncertainties associated with the modeling methods and data utilized in designing a nuclear reactor such as a beryllium-reflected, highly-enriched-uranium (HEU)-O2 fission surface power (FSP) system for space nuclear power. The conclusion of the previous study was that current capabilities could preclude the necessity of a cold critical test of the FSP; however, additional testing would reduce uncertainties in the beryllium and uranium cross-section data and the overall uncertainty in the computational models. A series of critical experiments using HEU metal were performed in the 1960s and 1970s in support of criticality safety operations at the Y-12 Plant. Of the hundreds of experiments, three were identified as fast-fission configurations reflected by beryllium metal. These experiments have been evaluated as benchmarks for inclusion in the International Handbook of Evaluated Criticality Safety Benchmark Experiments (IHECSBE). Further evaluation of the benchmark experiments was performed using the sensitivity and uncertainty analysis capabilities of SCALE 6. The data adjustment methods of SCALE 6 have been employed in the validation of an example FSP design model to reduce the uncertainty due to the beryllium cross section data.

Bess, John↗

The IAEA Coordinated Research Program on HTGR Reactor Physics, Thermal-hydraulics and Depletion Uncertainty Analysis: Description of the Benchmark Test Cases and Phases

The continued development of High Temperature Gas Cooled Reactors (HTGRs) requires verification of design and safety features with reliable high fidelity physics models and robust, efficient, and accurate codes. The uncertainties in the HTR analysis tools are today typically assessed with sensitivity analysis and then a few important input uncertainties (typically based on a PIRT process) are varied in the analysis to find a spread in the parameter of importance. However, one wish to apply a more fundamental approach to determine the predictive capability and accuracies of coupled neutronics/thermal-hydraulics and depletion simulations used for reactor design and safety assessment. Today there is a broader acceptance of the use of uncertainty analysis even in safety studies and it has been accepted by regulators in some cases to replace the traditional conservative analysis. Finally, there is also a renewed focus in supplying reliable covariance data (nuclear data uncertainties) that can then be used in uncertainty methods. Uncertainty and sensitivity studies are therefore becoming an essential component of any significant effort in data and simulation improvement. In order to address uncertainty in analysis and methods in the HTGR community the IAEA launched a Coordinated Research Project (CRP) on the HTGR Uncertainty Analysis in Modelling early in 2012. The project is built on the experience of the OECD/NEA Light Water Reactor (LWR) Uncertainty Analysis in Best-Estimate Modelling (UAM) benchmark activity, but focuses specifically on the peculiarities of HTGR designs and its simulation requirements. Two benchmark problems were defined with the prismatic type design represented by the MHTGR-350 design from General Atomics (GA) while a 250 MW modular pebble bed design, similar to the INET (China) and indirect-cycle PBMR (South Africa) designs are also included. In the paper more detail on the benchmark cases, the different specific phases and tasks and the latest status and plans are presented.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

RESULTS FOR THE INTERMEDIATE-SPECTRUM ZEUS BENCHMARK OBTAINED WITH NEW 63,65Cu CROSS-SECTION EVALUATIONS

The four HEU, intermediate-spectrum, copper-reflected Zeus experiments have shown discrepant results between measurement and calculation for the last several major releases of the ENDF library. The four benchmarks show a trend in reported C/E values with increasing energy of average lethargy causing fission. Recently, ORNL has made improvements to the evaluations of three key isotopes involved in the benchmark cases in question. Namely, an updated evaluation for 235U and evaluations of 63,65Cu. This paper presents the benchmarking results of the four intermediate-spectrum Zeus cases using the three updated evaluations.

Sobes, Vladimir↗