Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,747 records · Page 97
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

AGR 1 and 2 Isotopic Depletion Validation

This presentation includes work performed developing a computational benchmark for the AGR 1 and 2 experiments. These experiments involve irradiation of TRISO particle compact in the Advanced Test Reactor (ATR). Many post-irradiation measurements were made from these experiments, including isotopic concentrations of fission products. The benchmark involves a MCNP-ORIGEN coupled model of the ATR and AGR experiment capsules to calculate a comprehensive inventory of fission product and actinide concentrations in the compacts. This presentation describes the experiments, the measurements obtained, the benchmark model, and results of the model evaluation.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Data Driven Commercial Building Energy Code Compliance and Technology Inventory for New York City

Building Performance Standards (BPS) are gaining national traction. A BPS will require new processes in the design, construction, and operation of buildings that take the occupants into account and enable predictive analysis to ensure compliance with current and future GHG emissions caps. In New York City, most buildings over 25,000 square feet will be regulated by a BPS starting in 2024, regardless of whether it is new construction permitted under current energy codes or an existing building. This research is one of the first to begin the evaluation of a long-term series of building policies in the context of an open data ecosystem, in cooperation with city agencies. Existing building policies enacted in NYC have ranged from building energy benchmarking and labeling to energy audits to the regulation of GHG emission in buildings. Through the development of a dataset related to building technologies and energy consumption, this project can help to evaluate if meaningful conclusions can be drawn for the data that has been largely self-reported in compliance with city regulations. This project will also provide lessons learned from a deep dive into these types of datasets to provide best practices for municipalities or states seeking to embark on policies like those enacted in NYC. In addition, a Building Automation System (BAS) Stretch Standard of Care (SSOC) for owners, designers, and building operators will enable the measurement and predictive analysis of energy consumption and GHG emissions at the plant, system, or component level, in anticipation of regulated GHG limits on buildings based on energy use. The SSOC is expected to be suitable for use on a national level. The primary feature of an SSOC is a standardized format for a set of BAS points that can be used to control and to gather data from individual plants, systems, or components that are related to building energy consumption. This project examined how measurements compare to prescriptive or simulation-based energy code targets, finding little correlation between predictive 8760-hour energy modeling and actual energy consumption for a small sample (n=27) of buildings constructed after 2015. Other analysis found that, while large multifamily housing (MFH) buildings showed a general trend similar to predicted reductions in energy use from the implementation of model commercial energy codes, this trend was not evident in the office, K-12 school, and hotel use groups in NYC. No upward or downward trends in energy consumption were found when buildings were grouped by size. Energy audit data were analyzed and it appears that there is bias by audit company on measures recommended to clients. Further research should be performed to cross-analyze this with other attributes, such as building size, vintage, and number of stories. Analysis found that for 281 buildings that were permitted and completed after 2015 and had submitted benchmarking data in 2022, between 81% and 96% (by use group) were found to be in compliance with the 2024 to 2029 NYC BPS emission caps, and between 55% and 89% were in compliance with the 2030-2034 caps. This work is beneficial to the public in helping policymakers and building stakeholders better understand the wide-ranging implications of a BPS.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

An independent analysis of bias sources and variability in wind plant pre-construction energy yield estimation methods

The wind resource assessment community has long had the goal of reducing the bias between wind plant pre-construction energy yield assessment (EYA) and the observed annual energy production (AEP). This comparison is typically made between the 50% probability of exceedance (P50) value of the EYA and the long-term corrected operational AEP (hereafter OA P50), and is known as the P50 bias. The industry has critically lacked an independent analysis of bias reduction investigated across multiple consultants to identify the greatest sources of uncertainty and variance in the EYA process and the best opportunities for uncertainty reduction. The present study addresses this gap by benchmarking consultant methodologies against each other and against operational data at a scale not seen before in industry collaborations. We consider data from 10 wind plants and evaluate discrepancies between eight consultancies in the steps taken from estimates of gross to net energy. Consultants tend to overestimate the gross energy produced at the turbines and then compensate by further overestimating downstream losses, leading to a mean P50 bias near zero, still with significant variability among the individual wind plants. Within our data sample, we find that consultant estimates of all loss categories, except environmental losses, tend to reduce the project-to-project variability of the P50 bias. The disagreement between consultants, however, remains flat throughout the addition of losses. Finally, we find that differences in consultants’ estimates of project performance can lead to differences up to $10/MWh in the levelized cost of energy for a wind plant.

Todd, Austin C.↗
DOE OSTI2024

Phase Composition and Phase Transformation of Additively Manufactured Nickel Alloy 718 AM Bench Artifacts

Additive manufacturing (AM) technologies offer unprecedented design flexibility but are limited by a lack of understanding of the material microstructure formed under their extreme and transient processing conditions and its subsequent transformation during post-build processing. As part of the 2022 AM Bench Challenge, sponsored by the National Institute of Standards and Technology, this study focuses on the phase composition and phase evolution of AM nickel alloy 718, a nickel-based superalloy, to provide benchmark data essential for the validation of computational models for microstructural predictions. Here, we employed high-energy synchrotron X-ray diffraction, in situ synchrotron X-ray scattering, as well as high-resolution transmission electron microscopy for our analyses. The study uncovers critical aspects of the microstructure in its as-built state, its transformation during homogenization, and its phase evolution during subsequent aging heat treatment. Specifically, we identified secondary phases, monitored the dissolution and coarsening of microstructural elements, and observed the formation and stability of γ ’ and γ ” phases. The results provide the rigorous benchmark data required to understand the atomic and microstructural transformations of AM nickel alloy 718, thereby enhancing the reliability and applicability of AM models for predicting phase evolution and mechanical properties.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Advances in building data management for building performance standards using the SEED platform

Reducing energy consumption and greenhouse gas emissions in the built environment is a critical step in achieving emission goals to mitigate climate change impacts. Local, federal, and international jurisdictions are deploying several methods to reduce energy and emissions such as voluntary and mandatory benchmarking and building performance standards, requiring building owners to reach energy and emission targets. Jurisdictions leveraging benchmarking and building performance standards require knowledge of the buildings covered; which is a large task due to staffing constraints, limited information on building characteristics and tax parcel data, and the need for advanced data management techniques to align datasets. This paper describes an open-source platform's recent advances to create consistent taxonomies, identify erroneous data, enable auditability, and track building performance. The paper concludes with two use cases on how the platform has been used by jurisdictions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Radiation Oncology Resident Research Productivity in the United States: 2015 to 2019

Multiple efforts have been made in recent years to establish national benchmarks for research productivity among US radiation oncology residents. Morgan et al found a mean of 1.01 first-author, PubMed-searchable articles published by US radiation oncology residents over 4 years of residency between 2002 and 2007, whereas Verma et al found a mean of 1.97 first-author, PubMed-searchable articles published by members of the graduating US radiation oncology residency classes of 2014 and 2015. In this study, we sought to establish new national benchmarks for US radiation oncology resident research productivity and characterize the scholarly work produced by graduating US radiation oncology residents.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2024

Democratizing uncertainty quantification

Uncertainty Quantification (UQ) is vital to safety-critical model-based analyses, but the widespread adoption of sophisticated UQ methods is limited by technical complexity. In this paper, we introduce UM-Bridge (the UQ and Modeling Bridge), a high-level abstraction and software protocol that facilitates universal interoperability of UQ software with simulation codes. It breaks down the technical complexity of advanced UQ applications and enables separation of concerns between experts. UM-Bridge democratizes UQ by allowing effective interdisciplinary collaboration, accelerating the development of advanced UQ methods, and making it easy to perform UQ analyses from prototype to High Performance Computing (HPC) scale. In addition, we present a library of ready-to-run UQ benchmark problems, all easily accessible through UM-Bridge. These benchmarks support UQ methodology research, enabling reproducible performance comparisons. We demonstrate UM-Bridge with several scientific applications, harnessing HPC resources even using UQ codes not designed with HPC support.

Benchmarks↗
DOE OSTI2025

High-Fidelity CFD Assessments of Flow Resistance in a 61-Pin Wire-Wrapped Assembly with Partially Blocked Channels

The examination of thermal-hydraulic behaviors in wire-wrapped rod bundles continues to be an active area of research. The sodium fast reactor, a prominent candidate in next-generation nuclear designs, utilizes a hexagonal configuration of wire-wrapped fuel pins. Here, the potential for channel blockage within this compact arrangement poses a significant safety challenge, spurring a number of recent experimental and computational investigations to evaluate its impact on flow and heat transfer. The present work aims to benchmark the high-fidelity NekRS computational fluid dynamics (CFD) solver in predicting the pressure drops associated with substantial blockages, using available experimental data as a reference. A 61-pin wire-wrapped fuel assembly with two flow blockage configurations has been simulated and investigated at a range of low to moderate Reynolds numbers (487 ≤ Re ≤ 14 600). The NekRS solver demonstrates an exponential reduction of spatial discretization error with increasing polynomial order. The high level of agreement between the numerical results and measured data confirms the accuracy and consistency of the present numerical approach. This benchmark study establishes the capability of NekRS to perform reliable hydrodynamic simulations for sodium fast reactor applications and supports its use in design, licensing, and safety analyses.

CFD Benchmarking↗
DOE OSTI2020

Evaluation of simulated soil carbon dynamics in Arctic-Boreal ecosystems

Given the magnitude of soil carbon stocks in northern ecosystems, and the vulnerability of these stocks to climate warming, land surface models must accurately represent soil carbon dynamics in these regions. We evaluate soil carbon stocks and turnover rates, and the relationship between soil carbon loss with soil temperature and moisture, from an ensemble of eleven global land surface models. We focus on the region of NASA's Arctic-Boreal vulnerability experiment (ABoVE) in North America to inform data collection and model development efforts. Models exhibit an order of magnitude difference in estimates of current total soil carbon stocks, generally under- or overestimating the size of current soil carbon stocks by greater than 50 PgC. We find that a model's soil carbon stock at steady-state in 1901 is the prime driver of its soil carbon stock a hundred years later—overwhelming the effect of environmental forcing factors like climate. The greatest divergence between modeled and observed soil carbon stocks is in regions dominated by peat and permafrost soils, suggesting that models are failing to capture the frozen soil carbon dynamics of permafrost regions. Using a set of functional benchmarks to test the simulated relationship of soil respiration to both soil temperature and moisture, we find that although models capture the observed shape of the soil moisture response of respiration, almost half of the models examined show temperature sensitivities, or Q10 values, that are half of observed. Significantly, models that perform better against observational constraints of respiration or carbon stock size do not necessarily perform well in terms of their functional response to key climatic factors like changing temperature. This suggests that models may be arriving at the right result, but for the wrong reason. The results of this work can help to bridge the gap between data and models by both pointing to the need to constrain initial carbon pool sizes, as well as highlighting the importance of incorporating functional benchmarks into ongoing, mechanistic modeling activities such as those included in ABoVE.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Dual-map framework for noise characterization of quantum computers

In order to understand the capabilities and limitations of quantum computers, it is necessary to develop methods that efficiently characterize and benchmark error channels present on these devices. In this paper, we present a method that faithfully reconstructs a marginal (local) approximation of the effective noise (MATEN) channel, that acts as a single layer at the end of the circuit. We first introduce a dual-map framework that allows us to analytically derive expectation values of observables with respect to noisy circuits. These findings are supported by numerical simulations of the quantum approximate optimization algorithm (QAOA) that also justify the MATEN, even in the presence of nonlocal errors that occur during a circuit. Finally, we demonstrate the performance of the method on Rigetti's Aspen-11 quantum computer for QAOA circuits up to six qubits, successfully predicting the observed measurements on a majority of the qubits.

Sud, James↗
DOE OSTI2025

A Survey on the Design, Detection, and Prevention of Pre-Silicon Hardware Trojans

The complexity of the semiconductor design lifecycle and globalized manufacturing process creates concern over the threat of deliberate malicious alterations, or hardware Trojans, being inserted into microelectronic designs. This has resulted in a significant corpus of hardware Trojan research including Trojan design and benchmarking efforts and development of corresponding metrics and detection and prevention techniques, over the last two decades. In this survey, we first highlight efforts in Trojan design and benchmarking, followed by a cataloging of seminal and recent works in Trojan detection and prevention and their accompanied metrics. Given the volume of literature in this field, this survey considers only pre-silicon techniques. We make this distinction between pre- and post-silicon to properly scope and provide appropriate context into the capabilities of existing hardware Trojan literature. Each major section (design, prevention, and detection) is accompanied by insights, and common pitfalls, which we highlight can be addressed by future research.

42 ENGINEERING↗
DOE OSTI2026

SpaceNet 9—Cross-Sensor Alignment of Optical and SAR Imagery

Precise registration of high-resolution synthetic aperture radar (SAR) and optical imagery is necessary for realizing the full potential and benefits of multimodal image analysis. However, two significant challenges presently exist. First, there is a lack of annotated datasets and benchmarks available for high-resolution SAR–optical image registration. Second, an assessment of efficient and reliable image registration methods that can precisely align these modalities is lacking. Here, we present a holistic description of the SpaceNet 9 Challenge and its results. We present a description of the dataset and baseline algorithm along with the results of the challenge, including a description of the winning algorithms. We release the SpaceNet 9 dataset along with open-sourcing the winning algorithms and baseline. The objective of SpaceNet 9 was to compute a dense displacement map that indicates the shift needed to align pixels in an optical image to the pixels in a SAR image. The challenge launched in April 2025 and was active for approximately two months. The top five solutions reduced image alignment error from approximately 34 m to under 13 m for public and private test data, with the best results obtaining a registration error of only 8.5 and 6.7 m on the public testing and private testing dataset, respectively. Usage of pretrained image matching models, robust outlier rejection with RANSAC, and estimating local displacement were common among the top solutions. The results of this challenge provide insight into high-resolution SAR–optical image registration and offer opportunities for future benchmarking in this domain. The baseline algorithm, winning solutions, and datasets are available at https://spacenet.ai/sn9-challenge/.

benchmark datasets↗
DOE OSTI2021

Leveraging observed soil heterotrophic respiration fluxes as a novel constraint on global-scale models

Microbially-explicit models may improve understanding and projections of carbon dynamics in response to future climate change, but their fidelity in simulating global-scale soil heterotrophic respiration (RH), a stringent test for soil biogeochemical models, has never been evaluated. We used statistical global RH products, as well as 7,821 daily site-scale RH measurements, to evaluate the spatio-temporal performance of one first-order decay model (CASA-CNP) and two microbially-explicit biogeochemical models (CORPSE and MIMICS) that were forced by two different climate datasets. CORPSE and MIMICS did not provide any measurable performance improvement; instead, the models were highly sensitive to the meteorological input data used to drive them. Spatial RH variability was generally well simulated except in the northern middle latitudes (~50°N) and arid regions; models captured the seasonal variability of RH well, but showed more divergence in tropic and arctic regions. Our results demonstrate that the next generation of biogeochemical models shows promise, but also needs to be improved for realistic spatio-temporal variability of RH. Finally, we emphasize the importance of net primary production, soil moisture, and soil temperature inputs, and that jointly evaluating soil models for their spatial (global scale) and temporal (site scale) performance provides crucial benchmarks for improving biogeochemical models.

Jian, Jinshi↗
DOE OSTI2022

VERA BWR progression problems

During the first phase of the Consortium for Advanced Simulation of Light Water Reactors (CASL) program, the Virtual Environment for Reactor Applications (VERA) was developed with a focus on capabilities for high-fidelity, multiphysics simulation of pressurized water reactors (PWRs). During this development effort, a set of progression problems was created ranging from smaller pin cell calculations to larger 3D full-core calculations. These progression problems helped to guide the development of the software and served as benchmarks against which to test VERA. Since 2019, efforts have been made to extend the capabilities of VERA to model boiling water reactors (BWRs). Because BWR simulations come with many unique challenges, a set of BWR progression problems was developed to aid in this new effort. The BWR progression problems range from 2D lattice calculations to 3D mini-core problems, and reference neutronic solutions were computed using continuous-energy Monte Carlo codes. MPACT, one of the neutronics code in VERA, was benchmarked using the BWR progression problems. The code is capable of computing solutions to all problems. The eigenvalues computed by MPACT agree well with the Monte Carlo reference solutions. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

Sensitivity-based Experiment Design Optimization for a Molybdenum Critical Experiment

A lack of intermediate molybdenum benchmarks in the ICSBEP has been identified by LANL, Y-12, and IRSN. This lacking adversely effects criticality safety operations and leaves new differential molybdenum data unvalidated. NCERC is proposing a series of intermediate integral experiments to better the understanding of molybdenum systems. Using MCNP6.2 with the ENDF/B-VIII.0 nuclear data library a single unmoderated and four moderated system designs were identified using a sensitivity optimization method. Each proposed system was found to be at least twice as sensitive to the 95 Mo capture cross section in the URR as the sole existing intermediate molybdenum benchmark in the ISCBEP handbook. The addition of a new molybdenum sensitive intermediate system would improve future nuclear data evaluations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Multigroup Examination of Nickel-Reflected HEU System

Critical configurations of HEU-MET-FAST-003 (HMF-003) from the International Criticality Safety Benchmark Evaluation Project Handbook were modeled for addition into the Verified, Archived Library of Inputs and Data maintained at Oak Ridge National Laboratory. HMF-003 contains models of a highly enriched uranium sphere with a spherical metal reflector of natural uranium, tungsten carbide, or nickel. For these simple models, good agreement is expected between the k eff calculated with continuous-energy (CE) and multigroup (MG) transport. However, using the CSAS5 sequence of the SCALE 6.2.4 package, the 8 in. thick nickel reflector model resulted in a difference of around 1.2 % Δ k . Results presented a clear indication of poor performance for the SCALE 252-group based on ENDF/B-VII.1 cross section library for this model. The bias was investigated over a range of nickel thickness and then compared with other MG libraries available in SCALE 6.2.4. As the nickel thickness increases, the MG k eff deviates from the CE result, confirming a performance issue with the MG calculation. Additionally, the reactions were compared between the two libraries to help determine the cause of k eff bias. Finally, additional MG libraries within SCALE 6.3 and based on ENDF/B-VII.1 and ENDF/B-VIII were reviewed. The 302-group and 1597-group result in significant improvements compared with the 252-group library. Nickel cross sections were modified from ENDF/B-VII.1 to ENDF/B-VIII. For the ENDF/B-VIII library, the deviation between the CE libraries and MG libraries is less pronounced than for ENDF/VII.1, except for the 1597-group. This examination clearly indicates the importance of validating calculations with applicable benchmark experiments and caution when using MG libraries.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Design of Critical Experiments Targeting Epithermal Cross Sections of Tantalum

Sandia National Laboratories (SNL) and Oak Ridge National Laboratory (ORNL) are collaborating to develop a capability to test the epithermal/intermediate cross sections of tantalum (Ta) at the SNL critical experiments facility using the Seven Percent Critical Experiment (7uPCX) fuel. The Sandia Critical Experiments (SCX) Program provides a specialized facility for performing water moderated and reflected critical experiments with UO 2 fuel rod arrays. A history of safe reactor operations and flexibility in reactor core configuration has resulted in the completion of several benchmark critical experiment evaluations that are published in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook. The critical experiments described here are scheduled to be performed at SCX in fiscal year 2023. Currently, fabrication details for the needed new equipment are being finalized for completion of the procurement process. The experiments will then be executed and documented for publication consideration by the ICSBEP.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Impact of Thermal Scattering Law on Similarity Assessment in Light-Water or Polyethylene-Moderated Systems

A collaborative effort between Pacific Northwest National Laboratory (PNNL) and Oak Ridge National Laboratory (ORNL) is underway to provide a technical basis and methodology for the criticality safety community to use the sum of fractions (SoF) method for generating limits for mixtures of “selected actinide nuclides” included in the ANSI/ANS 8.15 standard. The PNNL scope in this project is to define a range of mixtures of 233 U, 235 U, and 239 Pu moderated with either light water or polyethylene and to examine the critical masses for these mixtures. The ORNL scope is primarily to provide validation support for these studies. More complete discussion of the project and its validation aspects will be presented at the upcoming International Conference on Nuclear Criticality Safety (ICNC) this Fall in Sendai, Japan. Clear differences in benchmark similarity to application systems as assessed by the integral parameter ck are noted in the validation studies performed as part of this project as a function of moderator. The c k value is a correlation coefficient that represents that amount of shared uncertainty in k eff due to cross sections between two systems. Individual nuclide-reaction contributions between the two systems can be simply summed to arrive at the total c k value. Specifically, the c k values for light-water–moderated solution experiments are higher for a water-moderated application than for a polyethylene-moderated application. This result is neither totally unexpected nor surprising, but the magnitude of the difference was difficult to anticipate. The TSUNAMI sequence, in the SCALE 6.2.4 code package developed by ORNL, was used to generate eigenvalues and reactivity effects with perturbation-theory based approach through sensitivity coefficients for all nuclides in the system with all reactions and energy groups. The TSUNAMI-Indices and Parameters (IP) sequence then uses the sensitivity data generated through TSUNAMI to generate relational parameters (i.e., c k ) to determine the degree of similarity between systems. One detail of the SCALE material and data implementation must be discussed at this point. Several thermal scattering laws (TSLs) are available for 1 H. SCALE uses a different nuclide ID number for each TSL; essentially, each version of 1 H is treated as a unique nuclide. For example, 1 H bound in water ( 1 H-H2O) is assigned the nuclide ID 1001, whereas 1 H bound in polyethylene (h-poly) is assigned the nuclide ID 9001001. The same cross section data are used for all reactions in 1 H, regardless of TSL, except for scattering below the TSL cutoff energy. TSUNAMI-IP treats different nuclide IDs as different nuclides; thus, no uncertainty is shared between 1 H-H2O and h-poly, despite much of the same data, including covariance data, being used for both nuclides. This presents a question: how much of the difference in assessed similarity between water- and polyethylene-moderated systems is due to the moderators, and how much is caused by the treatment of 1 H-H2O and h poly with cross section and covariance data. The extended edits generated by TSUANMI-IP allow for an investigation of this issue specifically, as well as a demonstration of the general techniques available within TSUNAMI to understand the results of the similarity assessment. This paper presents and analyzes the similarity assessment of both water- and polyethylene-moderated systems for a single benchmark: PST-002-001.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗