Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

The Surface-Topography Challenge: A Multi-Laboratory Benchmark Study to Advance the Characterization of Topography

Surface performance is critically influenced by topography in virtually all real-world applications. The current standard practice is to describe topography using one of a few industry-standard parameters. The most commonly reported number is Ra, the average absolute deviation of the height from the mean line (at some, not necessarily known or specified, lateral length scale). However, other parameters, particularly those that are scale-dependent, influence surface and interfacial properties; for example the local surface slope is critical for visual appearance, friction, and wear. The present Surface-Topography Challenge was launched to raise awareness for the need of a multi-scale description, but also to assess the reliability of different metrology techniques. In the resulting international collaborative effort, 153 scientists and engineers from 64 research groups and companies across 20 countries characterized statistically equivalent samples from two different surfaces: a “rough” and a “smooth” surface. The results of the 2088 measurements constitute the most comprehensive surface description ever compiled. We find wide disagreement across measurements and techniques when the lateral scale of the measurement is ignored. Consensus is established through scale-dependent parameters while removing data that violates an established resolution criterion and deviates from the majority measurements at each length scale. Our findings suggest best practices for characterizing and specifying topography. The public release of the accumulated data and presented analyses enables global reuse for further scientific investigation and benchmarking.

42 ENGINEERING↗
DOE OSTI2019

Initial Benchmarking of the Liquid Sampling-Atmospheric Pressure Glow Discharge-Orbitrap System Against Traditional Atomic Mass Spectrometry Techniques for Nuclear Applications

The integration of the liquid sampling-atmospheric pressure glow discharge (LS-APGD) ion source with Orbitrap mass spectrometers has resulted in new opportunities in the field of isotope ratio mass spectrometry. In a field that has been dominated by thermal ionization mass spectrometry (TIMS) and inductively coupled plasma mass spectrometry (ICP-MS) on quadrupole and scanning-mode sector field analyzer platforms for highly accurate and precise measurements, the LS-APGD-Orbitrap system offers a benchtop instrument capable of meeting the rigorous International Target Values for measurement uncertainty for uranium (U). In order to benchmark the LS-APGD-Orbitrap, a series of U certified reference materials with increasing {sup 235}U isotopic composition were analyzed. By using U samples ranging in enrichment from 1 to 80%, the ability of the system to measure isotope ratios over a wide range is demonstrated. This analysis represents the first time that the LS-APGD-Orbitrap system has been used to analyze highly enriched U samples, allowing for the measurement of each of the U isotopes, including {sup 234}U and {sup 236}U-related species, which had not been achieved previously. Ultimately, the LS-APGD-Orbitrap system was able to measure CRM U-800 (assayed as {sup 235}U / {sup 238}U = 4.265622) as 4.266922, with a combined uncertainty, (u{sub c}), of 0.040%. These results are compared to those obtained using traditional elemental mass spectrometers including TIMS and ICP-MS-based instruments. The effectiveness of the LS-APGD-Orbitrap MS system for measuring U isotopes shows excellent promise in nuclear forensics, safeguards, and other nuclear weapon-based applications. .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

Validation Data for Benchmarking Wire Arc Additive Manufacturing Process Simulations

Residual stresses cause geometric distortion and affect mechanical performance of additively manufactured structures, yet they are notoriously difficult to assess and predict. Distortion (warpage) can drive parts outside dimensional tolerance limits, leading to part rejection or rework. For parts that meet tolerance, locked-in residual stress fields can affect structural integrity during operation, particularly subcritical cracking by fatigue, creep, or corrosion. This work develops benchmark data for a common additive manufacturing process (Wire Arc Additive Manufacturing) that can be applied for calibration and validation of physical process models that predict residual stress fields. The work includes design of two different samples of differing geometry, detailed manufacturing records for a set of physical samples, and an extensive set of residual stress measurement data developed using two diverse techniques (the contour method and neutron diffraction). An initial application of the work is also reported, where a modeling challenge was issued to secure residual stress model predictions from two independent laboratories that were blind to residual stress measurement data. These initial blind residual stress predictions show significant discrepancies relative to the measurement data, illustrating the potential value of the underlying validation data. An open repository for this work, including the sample designs, manufacturing process records, and the residual stress data, is also provided for future application in non-blind validation efforts.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Benchmark microgravity experiments and computations for 3D dendritic-array stability in directional solidification

In this study, we present a comprehensive quantitative analysis of stability bands for dendritic arrays during directional solidification of a transparent succinonitrile-0.46 wt % camphor alloy, spanning a broad range of pulling velocities. Taking advantage of the microgravity environment aboard the International Space Station where most convection effects are suppressed, we obtain unique measurements that quantify the stable primary spacing range of spatially extended three-dimensional dendritic array structures under purely diffusive growth conditions. Through carefully designed velocity jump experiments and detailed examination of sub-grain boundary dynamics, we characterize key instabilities, including elimination and tertiary branching, shedding new light on the mechanisms governing dynamic dendritic spacing selection in extended 3D arrays. Phase field simulations are performed to characterize the stability limits of dendritic array structures for quantitative comparison with the flight experiments. Although the simulations capture general trends, significant deviations are noted at the upper stability boundary, indicating the influence of additional, unexplored factors. These findings contribute to a deeper understanding of dendritic growth dynamics and offer valuable benchmark data that could aid in refining predictive models and improving control of dendritic microstructures in metallurgical applications.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Benchmark problems in aerosol evolution: Comparison of some exact and DSMC results

Aerosols are generated in many normal or accident situations associated with the nuclear enterprise. For a good understanding and modeling of the nuclear source term, for example, good experimental data and computational programs relating to aerosol evolution are needed. In the past several years there has been an effort to explore use of the Direct Simulation Monte Carlo (DSMC) approach for such estimations to improve fidelity of computations to the actual physics and chemistry of the accidents. An integral part of these efforts has been verification and validation of the DSMC technique against other available results wherever possible. Herein this paper explores verification of DSMC against one existing and two new benchmark problems covering condensation, coagulation, deposition and two-component aerosols. The simulations compare well with the exact results, providing further confidence in the use of DSMC.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2018

Validation of statistical uncertainties in subcritical benchmark measurements: Part II – Measured data

Part I of this work presented an uncertainty approach which incorporates the singles and doubles counting rates determined by the Hage-Cifarelli formalism of the Feynman Variance-to-Mean method. This moments-based approach utilizes time correlations between prompt neutrons detections to assess the system multiplication. Additionally, it presented a validation which utilized simulated data generated using a 0-D point-kinetics Monte Carlo code. In Part II, this same method and validation approach is applied to measured data of a subcritical benchmark experiment. These measurements were performed with a 4.5 kg sphere of alpha-phase weapons grade plutonium reflected by copper and/or polyethylene. Finally, the results of 17 configurations are shown including detailed validation for four of the configurations.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2019

Full core 3D simulation of the BEAVRS benchmark with OpenMOC

The Method of Characteristics (MOC) has seen wide interest in reactor physics because of its accuracy and efficiency in computing lattice physics problems. While most of its use has been in solving 2D problems, there has been recent interest in extending MOC to 3D in order to more accurately calculate 3D power distributions in LWRs. While the method is naturally extensible to 3D, it presents significant computational difficulties. OpenMOC, an open source MOC simulator, has been extended from 2D to 3D capabilities. Simulation results are presented in this study for the full core 3D simulation of the BEAVRS benchmark, adequately resolved in space and angle.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Benchmarking Cu/BEA and HBEA catalysts for high-octane gasoline synthesis

We distinguish rates at which carbon deposition occurs during initiation, rates at which catalytic centers are lost during deactivation, and paraffin-to-olefin ratio during propagation as benchmarks that distinguish 5 wt% Cu/H-BEA and H-BEA (Si/Al = 13.5) catalysts during dimethyl ether (DME) homologation in the presence of hydrogen. Studies that systematically vary initial DME contact time (210, 94, and 45 mol H+, initial s (mol C ) -1 ), DME pressure (4 and 22 kPa), and H 2 pressure (1, 24, and 48 kPa) reveal that Cu enables lower carbon deposition rates (on a per proton basis) in the induction period, increases the effluent paraffin-to-olefin ratio during propagation, and decreases the instantaneous site-loss yields by a factor of ~ 1.5-2x (moles of active sites lost per mole of DME) during termination sequences thus affecting the degree of product saturation and catalyst stability during DME homologation. Furthermore, these results provide mechanistic insights revealing the critical role of Cu in facilitating DME homologation to high value, high-octane gasoline-range hydrocarbons with higher cumulative turnovers than proton form H-BEA.

09 BIOMASS FUELS↗
DOE OSTI2016

Benchmarking of manganese oxide materials with CO oxidation as catalysts for low temperature selective oxidation

Here CO oxidation behavior of different manganese oxide materials was studied as a model probe reaction to benchmark catalytic activities that could be useful for other aerobic oxidations. K-OMS-2, which showed intermediate activity [Ea = 3.0 and 6.3 kcal/mol for K-OMS-2 (made in solvent free procedure or SF) and K-OMS-2 (made via hydrothermal route or REF) respectively], were chosen for metal dopant studies using microwave assisted hydrothermal synthesis. These were further characterized using X-ray diffraction (XRD), N 2 sorption, Raman spectroscopy, scanning electron microscopy (SEM), and carbon monoxide temperature programmed reduction (CO-TPR). Additional characterization was carried out for Ni-K-OMS-2 materials using atomic absorption spectroscopy (AAS) and thermogravimetric analysis-mass spectroscopy (TGA-MS). Doping K-OMS-2 with Ni resulted in a large increase in oxygen mobility and re-adsorption, and the moderate activation energy for CO oxidation (13. kcal/mol) by Ni-K-OMS-2(SF) suggests that Ni-doped materials are good candidates for low temperature selective oxidation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Benchmark of numerical modeling approaches on the systematic performance evaluation of wave energy converters

Different numerical modeling methods have been developed and applied to evaluate a variety of performance indicators of wave energy converters (WECs), including the power performance, structural loads, levelized cost of energy, etc. Based on the modeling fidelity, the commonly used numerical modeling approaches can be classified as linear modeling, weakly nonlinear modeling and fully nonlinear modeling approaches. Each method differs in accuracy and computational efficiency, making them suitable for different stages of WEC design. However, the selection of modeling approach could significantly impact evaluation outcomes. For instance, simplified linear models may underestimate structural loads or overestimate energy production in some operational conditions, potentially leading to less cost-effective designs. Given the widespread utilization of these models, it is essential to understand the uncertainties brought by them in performance evaluations. This work is dedicated to benchmarking different linear-potential-flow-based numerical models for evaluating the systematic performance of WECs. Three representative numerical modeling approaches are considered in this work, including linear frequency-domain modeling, statistically linearized spectral-domain modeling and Cummins equation-based nonlinear time-domain modeling. A generic point absorber WEC is considered as the research reference in this work, and different sea sites are taken into account. The numerical models are utilized to predict critical performance indicators, including power performance, the annual energy production, the capacity factor, the levelized cost of energy and the PTO fatigue loads. By comparing the results, this work identifies the uncertainties associated with different modeling approaches in evaluating WEC performance.

Fatigue↗
DOE OSTI2019

Performance of biologically inspired algorithms tuned on TiO 2 nanoparticle benchmark system

Biologically inspired optimization algorithms (BIOA), such as particle swarm optimization (PSO) and differential evolution (DE), are often applied to a specific problem with the tacit assumptions that each algorithm will have a unique set of “ideal” tuning parameters that provide “best” performance, and that, once tuned, two different optimization algorithms ought to exhibit distinct performance. This work tests those assumptions by systematically comparing PSO and DE applied to identify stable and metastable configurations of a benchmark system, a titania dimer Ti 2 O 4 , whose energy is described by a modified Matsui-Akaogi pairwise potential. The hypotheses (H) are formally expressed as: (H1) for both PSO and DE there exists a unique, optimal set of tuning parameters that maximize its performance on a given problem; (H2) when running two different BIOAs, one algorithm must demonstrate superior performance over the other. By identifying and studying an approximate Pareto-optimal set of tuning parameters for DE and PSO, H1 is shown to be strictly false for DE, and tentatively false for PSO, while H2 is shown to be false with respect to certain statistics. Here, we further investigate the correlations between key tuning parameters and algorithm performance. Our conclusion may be implemented to further optimize and develop BIOAs applied to materials systems.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Benchmarking ADMM in Nonconvex NLPs

Here, we study connections between the alternating direction method of multipliers (ADMM), the classical method of multipliers (MM), and progressive hedging (PH). The connections are used to derive benchmark metrics and strategies to monitor and accelerate convergence and to help explain why ADMM and PH are capable of solving complex nonconvex NLPs. Specifically, we observe that ADMM is an inexact version of MM and approaches its performance when multiple coordination steps are performed. In addition, we use the observation that PH is a specialization of ADMM and borrow Lyapunov function and primal-dual feasibility metrics used in ADMM to explain why PH is capable of solving nonconvex NLPs. This analysis also highlights that specialized PH schemes can be derived to tackle a wider range of stochastic programs and even other problem classes. Our exposition is tutorial in nature and seeks to to motivate algorithmic improvements and new decomposition strategies.

42 ENGINEERING↗
DOE OSTI2025

Benchmarking image processing techniques for porosity measurement in polymer additive manufacturing: Review and experimental analysis

An image processing workflow is proposed for porosity measurement in polymer additive manufacturing. Various techniques, including global and local thresholding, region growing, and K-means clustering, were applied to microscopic images of carbon fiber reinforced acrylonitrile butadiene styrene (CF-ABS) and benchmarked for their ability to accurately measure porosity. Global methods included Otsu, minimum error, iterative, and entropy-based thresholding, while local methods included Niblack, Bernsen, Sauvola, and Bradley-Roth algorithms. Artificial uneven illumination was introduced to test local adaptive thresholds. Results showed significant differences in porosity values across methods. Otsu, region growing, and K-means clustering excelled under uniform illumination, while Sauvola and Bradley-Roth performed better with uneven illumination. Comparison with X-ray computed tomography (XCT) revealed slightly lower porosity values (2.55 %) than optimized methods (2.73–2.79 %) due to XCT's lower resolution excluding smaller pores. While XCT offers finer pore detection, it limits sample volume and underestimates porosity due to spatial variation. Validation using artificial grayscale images with 5 % porosity confirmed that Otsu, Bradley-Roth, region growing, and Sauvola algorithms produced accurate results. Although tested on a single material system, these methods can be adapted to others with optimization. In conclusion, given XCT's high computational and time costs, this study highlights suitable image processing techniques as cost-effective alternatives for porosity analysis in polymer composites.

Additive manufacturing↗
DOE OSTI2020

Benchmark of the KGMf with a coupled Boltzmann equation solver

The Kinetic Global Model framework (KGMf) is an open-source general-purpose global model (spatially averaged) simulation code developed to explore the reaction kinetics and pathways in plasma discharge systems. It contains species continuity and electron energy balance equations, with a time-dependent evaluated electron energy distribution function (EEDF) for electron impact reactions. The EEDF is utilized to determine the rate coefficients for electron impact reactions, which can have profound impact on the temporal evolutions of plasma parameters. Previously, the EEDF was commonly assumed as Maxwellian or an analytical function of the effective electron temperature. In this work, the KGMf is coupled with a Boltzmann equation (BE) solver to self-consistently compute the EEDF. The EEDF evolution frequency is determined based on relative changes of the reduced electric field. The KGMf is benchmarked with the ZDPlasKin code based on high-pressure low-temperature argon plasma discharge cases. Additionally, the temporal evolutions of reduced electric field, electron temperature, EEDF, reaction rates, and species densities, are obtained and compared under different discharge conditions, showing good agreement between the KGMf and the ZDPlasKin simulations. The application of the KGMf for predicting breakdown times in high power microwave discharges is also presented, which shows qualitative agreement with particle-in-cell simulations. The KGMf can be further applied for more complicated plasma discharge systems, where the reaction kinetics are more intricate (e.g., plasma-assisted combustion systems).

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Benchmarking collective effects of electron interactions in a wiggler with OPAL-FEL

OPAL-FEL is a recently developed tool for the modeling of particle accelerators containing wigglers or undulators. It extends the well established 3D electrostatic particle-tracking code OPAL, by merging it with the finite-difference time-domain electromagnetic solver MITHRA. We present results of two benchmark cases where OPAL-FEL simulations are compared to experimental results. Both experiments concern electron beamlines where the longitudinal phase space is modulated with a short magnetic wiggler. Good agreement was found in both the space charge and radiation dominated regimes.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Verification and benchmarking relativistic electron beam transport through a background gas

It is necessary to establish confidence in high-consequence codes containing an extensive suite of physics algorithms in the regimes of interest. Verification problems allow code developers to assess numerical accuracy and increase confidence that specific sets of model physics were implemented correctly in the code. The two main verification techniques are code verification and solution verification. In this work, we present verification problems that can be used in other codes to increase confidence in simulations of relativistic beam transport. Specifically, we use the general plasma code EMPIRE to model and compare with the analytical solution to the evolution of the outer radial envelope of a relativistic charged particle beam. Additionally, we also outline a benchmark test of a relativistic beam propagating through a vacuum and pressurized gas cell, and present the results between EMPIRE and the hybrid code GAZEL. Further, we discuss the subtle errors that were caught with these problems and detail lessons learned.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Decomprolute is a benchmarking platform designed for multiomics-based tumor deconvolution

Tumor deconvolution is a reliable way to disentangle the diverse cell types that comprise solid tumors. To date, however, both the algorithms developed to deconvolve tumor samples, and the gold standard datasets used to assess the algorithms are geared toward the analysis of gene expression (e.g., RNA-seq) rather than protein levels in tumor cells. While gene expression is less expensive to measure, protein levels provide a more accurate view of immune markers. To facilitate the development as well as improve the reproducibility and reusability of multi-omic deconvolution algorithms, we introduce Decomprolute, a Common Workflow Language framework that leverages containerization to compare tumor deconvolution algorithms across multiomic data sets. Decomprolute incorporates the large-scale multiomic data sets produced by the Clinical Proteomic Tumor Analysis Consortium (CPTAC), which include matched mRNA expression and proteomic data from thousands of tumors across multiple cancer types to build a fully open-source, containerized proteogenomic tumor deconvolution benchmarking platform. The platform consists of modular architecture and it comes with well-defined input and output formats at each module. As a result, it is robust and extendable easily with additional algorithms or analyses. The platform is available for access and use at http://pnnl-compbio.github.io/decomprolute.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Physics vs structure: A systematic benchmark of learning strategies for multi-zone building thermal dynamics

Recent advances in physics-informed and data-driven machine learning promise improved thermal models for advanced building control, yet there is limited quantitative evidence on when added physics structure and architectural complexity are beneficial. Here, this work presents a systematic benchmark of five representative system identification methods for modeling multi-zone building thermal dynamics: linear state-space models, multi-layer perceptrons, neural state-space models, neural ordinary differential equations, and physically-consistent neural networks. The methods are evaluated across multiple data regimes and zone coupling strategies. Using a high-fidelity multi-zone commercial building emulator, we examine short-term and long-term prediction accuracy, computational efficiency, and ease of development. Our results reveal critical trade-offs between prediction performance, model complexity, and physical consistency. We demonstrate that decoupled, nonlinear black-box models consistently outperform coupled physics-constrained architectures in both predictive accuracy and out-of-distribution robustness in majority of the test cases for the building type considered in the study. Our findings quantify the cost of complexity in building thermal modeling and provide concrete, actionable, scenario-based guidelines for selecting model classes for control-oriented applications.

Building thermal modeling↗