Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “BENCHMARKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Dataset for Role of electron correlation on the adenine dimer interaction for non-equilibrium geometries: a benchmark Quantum Monte Carlo study

Datasets for the calculations reported in "Role of electron correlation on the adenine dimer interaction for non-equilibrium geometries: A benchmark Quantum Monte Carlo study" by L. Washburn, A. Sedova, P. R. C. Kent. J. Chem. Phys. (2026) 165 (5): 054118. https://doi.org/10.1063/5.0332651. Includes the molecular geometries, QMCPACK, PySCF, and ORCA inputs and outputs, analysis scripts and files needed to reproduce all the figures and tables.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Sensitivity Studies, Gap Analysis, and Benchmark Experiment Optimization for Reactor Physics and Criticality Safety Applications

Many new reactor designs, such as advanced reactors and micro reactors, have materials that lack nuclear data validation. This is also true for many other applications in criticality safety and global security. Both differential and integral experiments are needed to validate cross-section data. Without this, a user cannot have confidence in the predicted results of a radiation-transport code. This work describes an approach called ARCHIMEDES (Application Relevant Critical/Subcritical HEU/Pu-based Integral Measurements for Enhancing Data and Evaluating Sensitivities) to design new criticality experiments that have similar k eff cross-section sensitivities to an application of interest. This process involves simulations to generate cross-section sensitivities to a parameter of interest (such as k eff ), a gap analysis to determine which existing benchmarks are most similar to the application, and an experiment optimization. Recently, there has been a great deal of interest in the reactor physics community on advanced reactors, micro reactors, and accelerator driven systems (ADS). This work will apply the described method to specific examples in this area. The focus of this work will be on the sensitivity study and gap analysis, while future work will include experiment design.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Investigation of Delayed Neutron Sensitivities for Several ICSBEP Benchmarks using MCNP

The effective delayed neutron (β$_{eff}$) is a very important parameter for reactor and criticality applications. This parameter is equal to the difference in reactivity between delayed critical ($k_{eff}$ = 1, which requires both prompt and delayed neutrons to achieve criticality) and prompt critical ($k_p$ = 1, which requires only prompt neutrons to achieve criticality). This is often referred to as the delayed critical "window" (the region of criticality between delayed and prompt critical). β$_{eff}$ is a reactor kinetics parameters and depends on the nuclides in the system that undergo fission as well as the spectral characteristics of the system. Measurements of β$_{eff}$ have been performed for many criticality experiments. The EUCLID (Experiments Underpinned by Computational Learning for Improvements in nuclear Data) project at Los Alamos National Laboratory (LANL) aims to constrain nuclear data by using a suite of measurement types beyond $k_{eff}$. Our team has recently investigated the use of pulsed spheres for nuclear data validation. Several other measurement methods are also of interest, including β$_{eff}$ (investigated here) and reactivity coefficients (investigated in a separate work at this same meeting). One focus of our work is to determine if other methods are complimentary to the critical experiments already utilized for nuclear data validation. This is important because if a method has similar sensitivities then it will not be particularly useful for nuclear data validation as it will provide the same information as the critical experiments already being used. Here "similar" could refer to several characteristics, one being the shape of a sensitivity profile over energy. In the future, these methods will be utilized (with both existing and new experiments) in machine learning algorithms for nuclear validation, similar to what is currently done for criticality experiments. In order to use a measurement type for nuclear validation, it is necessary to obtain cross-section sensitivities for that parameter. This work looks at one approach to estimate β$_{eff}$ sensitivities by utilizing $k_{eff}$ sensitivities within Monte Carlo N-Particle ® Code Version 6.2. This is applied to several criticality benchmarks. Results are compared and the benefits and limitations of this approach are discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Generalized Bayesian Framework for Evaluation of Integral Benchmark Experiments

A recently published generalized Bayesian optimization framework has provided a way to retract any or all of the three common assumptions underlying the conventional Generalized Linear Least Squares (GLLS) optimization method based on the concepts introduced in Ref. [2]. These assumptions are: 1. Perfection: The model used for data evaluation and the prior probability distribution function (PDF) of generalized data are perfect. 2. Normality: The prior and posterior PDF are normal. 3. Linearity: The model is linear. In this work we outline how the framework in [1] could be directly adopted for improved evaluation of nuclear criticality integral benchmark experiments (IBEs) by: 1. Removing the first assumption alone by utilizing the concept of imperfections introduced in [1] to enable evaluation in the presence of discrepancies between the data and model or of missing covariance information by a GLLS method that will be seen as a generalization of the conventional GLLS method employed by the TSURFER code, and by 2. Removing the remaining two assumptions by implementing a Markov Chain Monte Carlo method for computation of the posterior PDF in the SAMPLER code, where TSURFER and SAMPLER are the uncertainty quantification (UQ) codes for IBEs in the SCALE code system based on the GLLS and the stochastic method, respectively. The graphic in Figure 1 categorizes the methods discussed in terms of the assumptions that they employ to determine posterior PDFs.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Preliminary Results on Bayesian Inverse UQ for OECD/NEA WPNCS Subgroup 14 Benchmark Exercise for Error Recovery and Experimental Coverage

The Organization for Economic Cooperation and Development (OECD) Working Party on Nucelar Criticality Safety (WPNCS) has proposed a benchmark exercise representative of neutronic behavior in criticality experiments. Here, the goal is to develop confidence in data assimilation techniques used to adjust nuclear data. Participants are given synthetic experimental models with associated measured data and asked to estimate the model parameters given the model and measurements as well as provide predictions for separate application models. In this work, we performed data assimilation using Bayesian inverse Uncertainty Quantification (UQ) with machine learning surrogate models to produce posterior parameter distributions for the requested parameters and posterior predictive distributions for the requested responses. Several experimental models are shown to insufficiently inform the posterior parameter distributions for the applications involved. However, given sufficient experimental data, posterior parameter estimates yielded reduced uncertainty in the response predictions of interest while covering the experimental data.

Bayesian Inference↗
DOE OSTI2021

Time-Based CAN Intrusion Detection Benchmark

Modern vehicles are complex cyber-physical systems made of hundreds of electronic control units (ECUs) that communicate over controller area networks (CANs). This inherited complexity has expanded the CAN attack surface by the injection of malicious messages that vary their time-based characteristics. To detect these malicious messages, time-based intrusion detection systems (IDS) have been proposed. However, time-based IDS are usually trained and tested on low-fidelity datasets with unrealistic labeled attacks. This makes difficult the task of evaluating, comparing, and validating IDS. Here we detail and benchmark four time-based IDS in a dataset with real and advanced attacks. We found that methods with strong assumptions regarding the distribution of inter-arrival times have lower performance than distribution agnostic based methods. In particular, distribution agnostic based methods outperform distribution based methods at least on $55\%$ in area under the precision-recall (AUC-PR) curve. Our results expand the body of knowledge of CAN time-based IDS by providing details of these methods and reporting their results when tested on datasets with real and advanced attacks. We describe limitations, open challenges, and how lessons learnt from this research can inform the design of deployable time-based IDS in modern vehicles.

Blevins, Deborah↗
DOE OSTI2024

DAISY Benchmark Performance Data

This repository contains the underlying data from benchmark experiments for Drifting Acoustic Instrumentation SYstems (DAISYs) in waves and currents described in "Performance of a Drifting Acoustic Instrumentation SYstem (DAISY) for Characterizing Radiated Noise from Marine Energy Converters" (https://link.springer.com/article/10.1007/s40722-024-00358-6). DAISYs consist of a surface expression connected to a hydrophone recording package by a tether. Both elements are instrumented to provide metadata (e.g., position, orientation, and depth). Information about how to build DAISYs is available at https://www.pmec.us/research-projects/daisy. The repository's primary content is three compressed archives (.zip format), each containing multiple MATLAB binary data files (.mat format). A table relating individual data files to figures in the paper, as well as the structure of each file, is included in the repository as a Word document (Data Description MHK-DR.docx). Most of the files contain time series information for a single DAISY deployment (file naming convention: [site]_DAISY_[Drift #].mat) consisting of processed hydrophone data and associated metadata. For a limited number of DAISY deployments, the hydrophone package was replaced with an acoustic Doppler velocimeter (file naming convention: [site]_DAISY_[Drift #]_ADV.mat). Data were collected over several years at three locations: (1) Sequim Bay at Pacific Northwest National Laboratory's Marine & Coastal Research Laboratory (MCRL) in Sequim, WA, the energetic tidal channel in Admiralty Inlet, WA (Admiralty Inlet), and the U.S. Navy's Wave Energy Test Site (WETS) in Kaneohe, HI. Brief descriptions of data files at each location follow. - MCRL - (1) Drift #4 and #16 contrast the performance of a DAISY and a reference hydrophone (icListen HF Reson), respectively, in the quiescent interior of Sequim Bay (September 2020). (2) Drift #152 and #153 are velocity measurements for a drifting acoustic Doppler velocimeter in in the tidally-energetic entrance channel inside a flow shield and exposed to the flow, respectively (January 2018). (3) Two non-standard files are also included: DAISY_data.mat corresponds to a subset of a DAISY drift over an Adaptable Monitoring Package (AMP) and AMP_data.mat corresponds to approximately co-temporal data for a stationary hydrophone on the AMP (February 2019). - Admiralty Inlet - (1) Drift #1-12 correspond to tests with flow shielded DAISYs, unshielded DAISYs, a reference hydrophone, and drifting acoustic Doppler velocimeter with 5, 10, and 15 m tether lengths between surface expression and hydrophone recording package (July 2022). (2) Drift #13-20 correspond to tests of flow shielded DAISYs with three different tether materials (rubber cord, nylon line, and faired nylon line) in lengths of 5, 10, and 15 m (July 2022). - WETS - (1) Drift #30-32 correspond to tests with a heave plate incorporated into the tether (standard configuration for wave sites), rubber cord only, and rubber cord, but with a flow shielded hydrophone (November 2022). (2) Drift #49-58 and Drift #65-68 correspond to measurements around mooring infrastructure at the 60 m berth where time-delay-of-arrival localization was demonstrated for different DAISY arrangements and hydrophone depths (November 2022).

16 TIDAL AND WAVE POWER↗
DOE OSTI2020

Alaskan carbon-climate feedbacks will be weaker than inferred from short-term manipulations: Alaskan Benchmark Data and Model runs

This submission aimed to assess differences in short-term step warming manipulations and long-term chronic response to climate change in Alaskan ecosystems. Briefly, climate warming is occurring fastest at high latitudes. Based on short-term field experiments, this warming is projected to stimulate soil organic matter decomposition, and promote a positive feedback to climate change. We show here that the tightly coupled, nonlinear nature of high-latitude ecosystems implies that short-term (< 10 year) warming experiments produce emergent ecosystem carbon stock temperature sensitivities inconsistent with emergent multi-decadal responses. We first demonstrate that a well-tested mechanistic ecosystem model accurately represents observed carbon cycle and active layer depth responses to short-term summer warming in four diverse Alaskan sites. We then show that short-term warming manipulations do not capture the non-linear, long-term dynamics of vegetation, and thereby soil organic matter, that occur in response to thermal, hydrological, and nutrient transformations belowground. Our results demonstrate significant spatial heterogeneity in multi-decadal Arctic carbon cycle trajectories and argue for more mechanistic models to improve predictive capabilities.The model used in the current study is available publicly (https://github.com/jinyun1tang/ECOSYS), and the current submission contains the python/ matlab codes for analyzing output from the model (includng a readme file to explain the codes). The benchmark data, also enclosed, was collected from a range of published and publicly available sources (extracted using GRABIT: https://www.mathworks.com/matlabcentral/fileexchange/7173-grabit). These sources describe warming induced changes in tundra/ boreal ecosystems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Model-ready benchmarks for NPP, ANPP, litter fluxes, and recruitment into the 1 cm dbh size class

The intended use of this dataset is to serve as an observational benchmark to evaluate model predictions of NPP, ANPP, litter fluxes, and recruitment at Barro Colorado Island, Panama. This dataset contains four CSV files and one text file. “Benchmarks-NPP-ANPP-R-L.csv” provides estimates of annual ecosystem-level reproductive litter flux (R), leaf litter flux (L), aboveground net primary productivity (ANPP), and net primary productivity (NPP) for 61 field plots throughout tropical, temperate, and boreal forest biomes. An additional 499 plots (n = 550) include estimates of just R, L, and R/L. Each row reports a distinct set of estimates for one sampling interval at one plot. “Metadata-Benchmarks-NPP-ANPP-R-L.csv” contains field descriptions for all data fields in “Benchmarks-NPP-ANPP-R-L.csv”. “References-Benchmarks-NPP-ANPP-R-L.txt” contains full references to the original studies used to produce the observations at each plot included in the data. “Benchmarks-Recruitment.csv” provides estimates of species-level recruitment rates into the 1 cm dbh size class at four CTFS-ForestGeo sites using methods that account for unobserved mortality of new recruits between census intervals (Kohyama et al., 2018). “Metadata-Benchmarks-Recruitment.csv” contains field descriptions for all data fields in “Benchmarks-Recruitment.csv”.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

2024 Water Splitting Technologies Benchmarking and Protocols Workshop

The sixth annual Advanced Water Splitting Pathways Benchmarking meeting was held on June 11-12, 2024 at the Arizona State University California Center- Los Angeles, CA. A total of 117 people participated (102 in person and 15 via Zoom). Attendance at most breakout sessions ranged from 10 - 25 attendees. The focus of many of the sessions was on developing plans to validate protocols written to date, defining future protocols to be written and aligning with international efforts. The plenary session provided perspectives on international activities in each technology area, as well as an overview of the ARCHES Hydrogen Hub.

08 HYDROGEN↗
DOE OSTI2025

Benchmarking the Use of BPM Quadrupole Moments to Measure Emittance

For the PIP-II program, transverse emittance in the Fermilab Booster must remain well controlled at higher bunch intensities. 4-plate beam position monitors (BPMs) have a small but measurable quadrupole moment, making it possible to infer transverse emittance. By compositing many BPMs together, it becomes possible to improve the quality of the quadrupole signal. The Fermilab Booster BPM system has been used to measure these quadrupole moments in the past year and derive emittances from them. Recent benchmarks show that the derived BPM emittances show similar emittance evolution and value to IPM and Multiwire data. This approach can both supplement and complement existing non-intercepting emittance monitors in accelerators.

Balcewicz, M. A. [Fermilab]↗
DOE OSTI2020

Curation of Ground-Truth Validated Benchmarking Datasets for Fault Detection & Diagnostics Tools

Fault detection and diagnostics (FDD) analytical tools for heating, ventilation and air conditioning (HVAC) systems represent one of the most active areas of smart building technology development. A diversity of techniques is used for FDD analytics, spanning physical models, black box, and rule-based approaches, and researchers continuously strive to develop improved algorithms. With FDD algorithm numbers now in the hundreds, there is a need for performance evaluation of these algorithms in order to assess improvements, improve costeffectiveness, and to prioritize investment in the further development of these technologies. A persistent challenge of FDD advance has been the lack of common datasets to benchmark the performance accuracy of FDD algorithms. This paper summarizes the successful curation of HVAC operational data, paired with validated ground-truth information regarding the presence and absence of faults. The current dataset, consisting of both simulation and experimental data, will evolve to include a larger set of HVAC systems with the objective of creating the largest publicly available dataset to be used by FDD developers, users, and researchers to compare and contrast performance accuracy across FDD algorithms, helping to drive improvements that will spur greater market adoption of FDD tools. Furthermore, in order to avoid previously observed issues with contributed datasets and ensure high quality and consistency of future submissions, the development of data validation and ground-truth assessment protocol is detailed in this study.

Casillas, Armando↗
DOE OSTI2025

DeepBench: A simulation package for physical benchmarking data

We introduce **DeepBench**, a python library that generates simple simulated image data from first principles, such as basic geometric shapes and astronomical objects. These data are highly valuable for developing (calibration, testing, and benchmarking) statistical and machine learning models because they make it possible to connect the final data product to physically interpretable inputs. This software includes tools to curate and store the datasets to maximize reproducibility.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

High-Resolution Simulations of Geological CO 2 Injection: Application to the SPE11 Benchmark

Geological carbon sequestration (GCS) will play a critical role in decarbonization and in facilitating the transition to clean energy systems. Because CO 2 is highly mobile, ensuring its safe and permanent injection into subsurface geological formations involves monitoring over larger spatial domains and longer time periods than is typical for hydrocarbon reservoirs. This can benefit from simulation tools capable of modeling key CO 2 trapping mechanisms, particularly those optimized for speed and scalability on high-performance computing systems. Using isothermal versions of the SPE11B and SPE11C benchmark cases, we conduct a mesh refinement study simulating CO 2 injection into kilometer-scale rock formations at centimeter resolution with the GEOS open-source simulation framework. We focus on how mesh refinement improves the accuracy of convective mixing in both 2D and 3D simulations. The computational costs associated with achieving a converged solution highlight the need for predictive upscaling techniques. A systematic performance scaling analysis—including both central processing unit (CPU) and graphics processing unit (GPU) architectures—complements the “Results” section.

Geosciences↗
DOE OSTI2020

Benchmarking Utility-Scale PV Operational Expenses and Project Lifetimes: Results from a Survey of U.S. Solar Industry Professionals

This paper draws on a survey of solar industry professionals and other sources to clarify trends in the expected useful life and operational expenditure (OpEx) of utility-scale photovoltaic (PV) plants in the United States. Solar project developers, sponsors, long-term owners, and consultants have increased project-life assumptions over time, from an average of ~21.5 years in 2007 to ~32.5 years in 2019. Current assumptions range from 25 years to more than 35 years depending on the organization; 17 out of 19 organizations surveyed or reviewed use 30 years or more. Levelized, lifetime OpEx estimates have declined from an average of ~$\$$35/kW DC -yr for projects built in 2007 to an average of ~$\$$17/kW DC -yr in 2019. Across 13 sources, the range in average lifetime OpEx for projects built in 2019 is broad, from $\$$13 to $\$$25/kW DC -yr. Operations and maintenance (O&M) costs—one component of OpEx—have declined precipitously in recent years, to $\$$5-8/kW DC -yr in many cases. Property taxes and land lease costs are highly variable across sites, but on average are—together—of similar magnitude. Other OpEx line items include security, insurance, and asset management. Given 2007-2009 values for not only project life and OpEx but also other drivers of the levelized cost of energy (LCOE, excluding the investment tax credit), the LCOE for utility-scale PV projects built from 2007 through 2009 averaged $\$$305 /MWh. Using 2019 values for all parameters yields an average LCOE of $\$$51 /MWh. The decline in LCOE from $\$$305 /MWh to $\$$51 /MWh was predominantly caused by reductions in up-front expenditures (and, to a much lesser extent, by changes in capacity factors, financing costs, and tax rates), but 9% ($\$$22 /MWh) of the overall decline is due to improvements in project life and OpEx. Project life extensions and OpEx reductions have had similarly sized impacts on LCOE over this period, at $\$$11 /MWh each. Had project life and OpEx not improved over the last decade, LCOE in 2019 would have instead been $\$$73 /MWh—43% higher. Given the limited quantity and comparability of previously available data on these cost drivers, the data and trends presented here may inform assumptions used by electric system planners, modelers, and analysts. The results may also provide useful benchmarks to the solar industry, helping developers and assets owners compare their expectations for project life and OpEx with those of their peers.

14 SOLAR ENERGY↗
DOE OSTI2020

Benchmark Solutions for Radiation Transport in Stochastic Media with Inhomogeneous Material Statistics

Accurately solving implicit Monte Carlo (IMC) thermal photon transport problems with mixed material cells is important in realistic applications. The production IMC package at LLNL treats mixed material cells arising from ALE remap and hydrodynamics using the same approximate model. The new Imp IMC thermal photon transport package currently under development has both a material interface reconstruction (MIR) algorithm and a Levermore-Pomraning (LP) stochastic medium algorithm for treating mixed material cells. Existing stochastic medium algorithms for treating mixed material cells in IMC lack a complete theoretical basis. The IMC LP algorithm implementation has been demonstrated to reproduce published deterministic LP solutions for the particular case of spatially homogeneous material statistics. Realistic simulations will include spatially inhomogeneous material statistics (material mean chord lengths). In a previous investigation, the LP-model for transport in binary stochastic media in rod geometry was generalized to accommodate spatially varying material chord lengths, i.e., the mixing statistics were allowed to be nonhomogeneous. Analytical solutions were obtained and used to produce a verifi cation suite for the Imp IMC Levermore-Pomraning implementation for different spatial variations of the chord lengths. However, the accuracy of the LP model when the mixing statistics are nonhomogeneous has not been assessed and leaves open the question of whether local accuracy is improved or further degraded when chord lengths are not uniform. This shortcoming is rectifi ed here by developing benchmark analytic solutions for transport in binary Markovian stochastic mixtures in rod geometry with nonhomogeneous mixing statistics, using spatially varying chord lengths considered in the previous investigation based on the LP model. Methods for sampling a nonhomogeneous Poisson process (NHPP) are first described and used to construct individual realizations of the binary mixtures in rod geometry. Analytic solutions are then obtained for the forward and backward directed fluxes on a given realization, now viewed as a deterministic medium with alternating layers of the two materials with known interface locations. Finally, material averaged scalar fluxes are obtained using these sampling schemes with spatially linear and quadratic chord lengths and used to assess the accuracy of the previously obtained LP-model results.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Goldsim Modeling of Vadose Zone Transport for E-Area Naval Reactor Component Disposal Areas: Model Description and Benchmarking

This report documents the development of a GoldSim® model of flow and radionuclide transport to the water table through the Naval Reactor Components Disposal Area (NRCDA) waste disposal sites and underlying vadose zones. The model is designed to be used for Monte Carlo uncertainty analysis in support of the E-Area Performance Assessment (PA). This report describes the model and shows results obtained from benchmarking the model to best-estimate deterministic results obtained using a PORFLOW model of NRCDA vadose zone transport. The PORFLOW model is three-dimensional while the GoldSim model is a simplified one-dimensional treatment. Nevertheless, the GoldSim model was able to accurately reproduce PORFLOW results with some adjustment to the nominal dispersion coefficient and vadose zone flow area used as “tuning” parameters. An example of the results obtained comparing GoldSim and PORFLOW calculation of releases of I-129, Tc-99, C-14 and Ni-59 from waste disposal containers at the 643-26E site is shown in Figure 1 below. For all of the test cases evaluated, GoldSim predicted peak concentrations within 6% of the PORFLOW values and peak times agreed within 8% with the majority of the results in better agreement. The close agreement between the two models provides confidence that GoldSim will give results accurately reflecting the behavior of releases from the NRCDA under off-normal operating conditions for sensitivity and uncertainty analysis

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Benchmark Specification of Advanced Burner Test Reactor

As an effort to assess the Argonne Reactor Computation (ARC) suite of fast reactor analysis codes, a numerical benchmark problem was developed using the reference 250 MWt Advanced Burner Test Reactor (ABTR) metallic core fueled with beginning of equilibrium cycle compositions (Chang et al., 2006). Material thermal expansion at operating condition was modeled by adjusting the hexagonal pitch, axial meshes, and the fuel and structure material densities appropriately. Irradiation swelling of metal fuel was considered, and the bond sodium was displaced into the lower part of fission gas plenum.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗