Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Demonstration of Optimal Benchmark Selection Website and Validation of the q c Coverage Metric Using HEU-SOL-THERM-013-003 Experiment

In the work documented in this interim report, the experiment selection toolkit web site was demonstrated and q C coverage metric methodology was validated for IEU-MET-FAST-002-001, MIX-COMP-THERM 004-004, and HEU-SOL-THERM-013-003 experiments. 𝑞 𝐶 is an information-theoretic measure based on mutual information that quantifies the ability of candidate benchmark experiments to reduce the bias and uncertainty of a target criticality safety application. The metric and an accompanying open-source Python toolkit with a web-based interface were tested against a benchmark set of 425 experiments drawn from the International Criticality Safety Benchmark Evaluation Project Handbook. The interface is hosted at https://edim.covdef.com. It accepts sensitivity data files produced by the TSUNAMI-IP module of the SCALE code system and supports both (i) deterministic analysis using the ENDF/B-VII.0 covariance library and (ii) stochastic analysis based on user-supplied keff samples. Demonstrations on representative applications across a range of material composition, spectrum, and form show that q C -guided benchmark selection achieves greater uncertainty reduction with fewer experiments and yields more stable posterior bias and uncertainty estimates than traditional similarity coefficient ( c k )–based selection, while also capturing valuable low-ck experiments that one-to-one metrics overlook.

Abdel-khalik, Hany S. [Indiana Univ.-Purdue Univ. ↗
DOE OSTI2024

Benchmarking MELCOR's NAC Package to ABCOVE Tests AB5 and AB6

This report presents analyses of the AB5 and AB6 ABCOVE sodium spray fire experiments with the MELCOR code. This code simulates the progression of accident events for analysis and auditing purposes of nuclear facilities during accident conditions. Historically, the ABCOVE experiments have contributed to the validation of aerosol physics and related phenomena. Given advancements in sodium-cooled reactor designs, characterization of the sodium spray combustion may further the review and validation of newly incorporated sodium properties and physics packages, namely, the sodium equations of state (EOS) and the sodium combustion (NAC) package within MELCOR. By analyzing the AB5 and AB6 experiments with and without the NAC package, sodium specificity for spray combustion and aerosol formation as well as speciation of the combustion products are reviewed with the new packages. This effort provides code users with a demonstration of the current code capabilities. This report provides the current best practices for the NAC package as well as a discussion of any issues observed while performing the presented analyses.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Benchmarking MELCOR's NAC Package to ABCOVE Test AB7

This report presents analyses of the AB7 ABCOVE sodium spray fire experiment with the MELCOR code. This code simulates the progression of accident events for analysis and auditing purposes of nuclear facilities during accident conditions. Historically, the ABCOVE experiments have contributed to the validation of aerosol physics and related phenomena. Given advancements in sodium-cooled reactor designs, characterization of the sodium spray combustion may further the review and validation of newly incorporated sodium properties and physics packages, namely, the sodium equations of state (EOS) and the sodium combustion (NAC) package within MELCOR. Previously, the AB5 and AB6 experiments were analyzed with and without the NAC package. This work builds on the previous analyses with a demonstration of the current code capabilities of MELCOR with a more mild Na spray and pool fire scenario.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

MS25: Materials Science-Focused Benchmark Data Set for Machine Learning Interatomic Potentials

Here, we present MS25, a benchmark data set for evaluating machine learning interatomic potentials (MLIPs) across diverse materials-relevant systems including MgO surfaces, liquid water, zeolites, a catalytic Pt surface reaction, high-entropy alloys (HEAs), and disordered Zr-oxides. Five MLIP architectures (MACE, NequIP, Allegro, MTP, and Torch-ANI) are trained and tested, focusing not only on traditional metrics (energies, forces, and stresses) but also explicitly validating derived physical observables such as lattice constants, volumes, and reaction barriers. We find that most models reach comparable accuracy on standard error metrics across the simple systems, although equivariant MLIPs offer 1.5–2× improvements over nonequivariant MLIPs in energy and force error for structurally complex or compositionally disordered environments such as HEAs and Zr–O systems. Our analysis highlights that low errors in energy and force predictions do not guarantee reliable observables, emphasizing the necessity of explicit validation. We demonstrate limitations in cross-framework transferability, as models trained on one zeolite framework (CHA) fail to reliably generalize to predictions of structurally distinct frameworks (e.g., MFI). Size-extensive tests show some dependence on system size for MgO, resulting from forced periodicity. The HEA and Zr–O data sets are identified as challenging tests for future benchmarks and MLIP model architecture developments as they show significant differentiation in error between MLIP architectures and are still relatively difficult at 1000 training images. Moving forward, we recommend that benchmarking efforts shift their focus from marginal accuracy improvements in energy and force errors toward identifying and understanding model failure modes, rigorously assessing transferability, and evaluating how their errors affect observable predictions. For researchers looking to choose an MLIP architecture, we suggest selecting equivariant MLIP architectures if the complexity of the system is a challenge. For simple materials problems, auxiliary features such as integration with molecular dynamics engines, trade-offs between computational data set generation cost vs MLIP inference speed, and framework integration may play a more important decision factor than small differences in error metrics that are unlikely to matter for production-level research.

chemical structure↗
DOE OSTI2022

Assessment of four strain energy decomposition methods for phase field fracture models using quasi-static and dynamic benchmark cases

Abstract Strain energy decomposition methods in phase field fracture models separate strain energy that contributes to fracture from that which does not. However, various decomposition methods have been proposed in the literature, and it can be difficult to determine an appropriate method for a given problem. The goal of this work is to facilitate the choice of strain decomposition method by assessing the performance of three existing methods (spectral decomposition of the stress or the strain and deviatoric decomposition of the strain) and one new method (deviatoric decomposition of the stress) with several benchmark problems. In each benchmark problem, we compare the performance of the four methods using both qualitative and quantitative metrics. In the first benchmark, we compare the predicted mechanical behavior of cracked material. We then use four quasi-static benchmark cases: a single edge notched tension test, a single edge notched shear test, a three-point bending test, and a L-shaped panel test. Finally, we use two dynamic benchmark cases: a dynamic tensile fracture test and a dynamic shear fracture test. All four methods perform well in tension, the two spectral methods perform better in compression and with mixed mode (though the stress spectral method performs the best), and all the methods show minor issues in at least one of the shear cases. In general, whether the strain or the stress is decomposed does not have a significant impact on the predicted behavior.

Zhang, Shuaifang↗
DOE OSTI2020

Call for Participation: Collaborative Benchmarking of Functional-Structural Root Architecture Models. The Case of Root Water Uptake

Three-dimensional models of root growth, architecture and function are becoming important tools that aid the design of agricultural management schemes and the selection of beneficial root traits. However, while benchmarking is common in many disciplines that use numerical models, such as natural and engineering sciences, functional-structural root architecture models have never been systematically compared. The following reasons might induce disagreement between the simulation results of different models: different representation of root growth, sink term of root water and solute uptake and representation of the rhizosphere. Presently, the extent of discrepancies is unknown, and a framework for quantitatively comparing functional-structural root architecture models is required. We propose, in a first step, to define benchmarking scenarios that test individual components of complex models: root architecture, water flow in soil and water flow in roots. While the latter two will focus mainly on comparing numerical aspects, the root architectural models have to be compared at a conceptual level as they generally differ in process representation. Therefore, defining common inputs that allow recreating reference root systems in all models will be a key challenge. In a second step, benchmarking scenarios for the coupled problems are defined. We expect that the results of step 1 will enable us to better interpret differences found in step 2. This benchmarking will result in a better understanding of the different models and contribute toward improving them. Improved models will allow us to simulate various scenarios with greater confidence and avoid bugs, numerical errors or conceptual misunderstandings. This work will set a standard for future model development.

Plant Sciences↗
DOE OSTI2023

SYNCA: A Synthetic Cyclotron Antenna for the Project 8 Collaboration

Cyclotron Radiation Emission Spectroscopy (CRES) is a technique for measuring the kinetic energy of charged particles through a precision measurement of the frequency of the cyclotron radiation generated by the particle's motion in a magnetic field. The Project 8 collaboration is developing a next-generation neutrino mass measurement experiment based on CRES. One approach is to use a phased antenna array, which surrounds a volume of tritium gas, to detect and measure the cyclotron radiation of the resulting β-decay electrons. To validate the feasibility of this method, Project 8 has designed a test stand to benchmark the performance of an antenna array at reconstructing signals that mimic those of genuine CRES events. To generate synthetic CRES events, a novel probe antenna has been developed, which emits radiation with characteristics similar to the cyclotron radiation produced by charged particles in magnetic fields. This paper outlines the design, construction, and characterization of this Synthetic Cyclotron Antenna (SYNCA). Furthermore, we perform a series of measurements that use the SYNCA to test the position reconstruction capabilities of the digital beamforming reconstruction technique. Finally, we find that the SYNCA produces radiation with characteristics closely matching those expected for cyclotron radiation and reproduces experimentally the phenomenology of digital beamforming simulations of true CRES signals.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Building fault detection data to aid diagnostic algorithm creation and performance testing

It is estimated that approximately 4-5% of national energy consumption can be saved through corrections to existing commercial building controls infrastructure and resulting improvements to efficiency. Correspondingly, automated fault detection and diagnostics (FDD) algorithms are designed to identify the presence of operational faults and their root causes. A diversity of techniques is used for FDD spanning physical models, black box, and rule-based approaches. A persistent challenge has been the lack of common datasets and test methods to benchmark their performance accuracy. This article presents a first of its kind public dataset with ground-truth data on the presence and absence of building faults. This dataset spans a range of seasons and operational conditions and encompasses multiple building system types. It contains information on fault severity, as well as data points reflective of the measurements in building control systems that FDD algorithms typically have access to. The data were created using simulation models as well as experimental test facilities, and will be expanded over time.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Discrete fracture network model benchmarks developed and applied in a DECOVALEX-2023 repository performance assessment study

This study presents newly developed benchmarks for modeling flow and transport within discrete fracture networks (DFNs) and useful methods for analyzing the results. The new benchmarks are designed to test modeling approaches for use in probabilistic performance assessment models of deep geologic repositories in fractured rock. The benchmarks simulate flow and transport through a 1 km 3 block of fractured rock. The first simulates migration of a short pulse of tracer through a simple network of four intersecting fractures. The second adds 1089 stochastically generated fractures. The third changes the pulse to a continuous point source. Evaluation of model performance relies on moment analysis and comparison of the results of different models. The expected nondimensional first moment of the conservative tracer for each benchmark is 1. The benchmarks were simulated by teams from Canada, Czechia, Germany, Korea, Sweden, Taiwan, and the United States as part of a DECOVALEX-2023 study (decovalex.org). The teams used various approaches, including explicit DFN modeling, DFN upscaling to an equivalent continuous porous medium (ECPM), and a combination of both methods. Transport mechanisms are modeled using either the advection-dispersion equation or particle tracking. Results demonstrate strong agreement among the models in breakthrough behavior up to the 75th percentile. Significant deviations in first moments and well-clustered outputs led to the identification of inaccuracies in several models. Such findings exemplify the benefit of exercising these benchmarks and using the presented methods to test DFN flow and transport models.

Benchmark↗
DOE OSTI2020

SO(3)-invariance of informed-graph-based deep neural network for anisotropic elastoplastic materials

This work examines the frame-invariance (and the lack thereof) exhibited in simulated anisotropic elasto-plastic responses generated from supervised machine learning of classical multi-layer and informed-graph-based neural networks, and proposes different remedies to fix this drawback. The inherent hierarchical relations among physical quantities and state variables in an elasto-plasticity model are first represented as informed, directed graphs, where three variations of the graph are tested. While feed-forward neural networks are used to train path-independent constitutive relations (e.g., elasticity), recurrent neural networks are used to replicate responses that depends on the deformation history, i.e. or path dependent. In dealing with the objectivity deficiency, we use the spectral form to represent tensors and, subsequently, three metrics, the Euclidean distance between the Euler Angles, the distance from the identity matrix, and geodesic on the unit sphere in Lie algebra, can be employed to constitute objective functions for the supervised machine learning. In this, the aim is to minimize the measured distance between the true and the predicted 3D rotation entities. Following this, we conduct numerical experiments on how these metrics, which are theoretically equivalent, may lead to differences in the efficiency of the supervised machine learning as well as the accuracy and robustness of the resultant models. Neural network models trained with tensors represented in component form for a given Cartesian coordinate system are used as a benchmark. Our numerical tests show that, even given the same amount of information and data, the quality of the anisotropic elasto-plasticity model is highly sensitive to the way tensors are represented and measured. The results reveal that using a loss function based on geodesic on the unit sphere in Lie algebra together with an informed, directed graph yield significantly more accurate rotation prediction than the other tested approaches.

42 ENGINEERING↗
DOE OSTI2021

Closing the Loop on Automotive Carbon Fiber Prepreg Manufacturing Scrap

The project demonstrated how to “close the loop” on carbon fiber by integrating industrial carbon fiber scrap into new functional components in an automotive lightweighting application. The project serves as a validation of discontinuous recycled carbon fiber in a commercial context, while generating comprehensive material data throughout the production chain. To this end, the project exhibited increasing complexity as material evaluation progressed from benchtop to commercial scale through full-scale part production, with key material properties thoroughly characterized throughout the process. Of particular focus was the form of the fiber that was fed into compounding, as recycled fiber has historically been problematic to feed at commercial-scale. Carbon fiber is energy-intensive to manufacture, so reuse of existing fiber material can reduce costs and increase sustainability. Additionally, by integrating recovered short fiber into a thermoplastic, regrind processes can be used to provide feedstock for later generations of product. While regrind plastics are not “infinitely recyclable”, reusing the manufacturing scrap over several generations of products can greatly increase material sustainability and lower the fractional embodied energy of each successive product. As such, this project supports the IACMI technical goals of (1) 25% lower carbon fiber-reinforced polymer (CFRP) cost, (2) 50% reduction in CFRP embodied energy, and (3) 80% composite recyclability into useful products. The initial stage of the project involved down selecting surface treatment (sizing) chemistries. Sizing evaluations were performed on Vartega’s chemically recycled intermediate modulus fiber along with standard modulus dry scrap which was oversized with sizing provided by Michelman. More dramatic improvements from sizing were found on the standard rather than the intermediate modulus fibers. The strength of the chemically recycled individual fibers were evaluated by Michelman and ORNL through single fiber testing and found to be comparable to similarly evaluated virgin fibers. UDRI’s mechanical testing on injection molded test specimens identified similar mechanical properties and fiber distribution relative to benchmark specimens. Additional surface chemistry tests and visualizations were performed by the Colorado School of Mines to confirm close conformance between the benchmark and recycled-fiber specimens. As the mechanical test results exceeded the 80% threshold established as the go/no-go(GNG), the project scale was increased to use commercial-scale equipment that would both better characterize the manufacturing utility of the target product format and allow qualitative assessment of a complex commercial part. An upscaled compounding evaluation was performed with a 27 mm twin-screw compounding extruding using oversized standard modulus fibers that were formatted to improve bulk solids transfer. The project team anticipated that milestone mechanical benchmarks could be achieved given the favorable performance of the sized standard modulus material identified in the initial micro-compounding trials. While the mechanical performance did meet the milestone target for that phase of the project, mechanical properties for this standard modulus-based compound were still less than those of the Ford specification. To compare the performance, the project team oversized intermediate modulus dry fibers and compounded them with the project resin at BASF using a 40 mm compounder. Test specimen mechanical performance exceeded the targets laid out in both the project milestone and the Ford specification. A series of prototype parts were successfully molded, albeit with instances of short shot components due to the high thermal conductivity of the carbon fiber compared to glass fiber for which the prototype tool was designed for. The project demonstrated that recycled carbon fiber is a viable option in fiber reinforced compound, providing greatly increased strength and modulus for applications that require them. The “agglomerated” format that facilitated effective bulk solids transfer of recovered fiber showed promise for industrial application.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Phase-field modeling of rate-dependent fluid-driven fracture initiation and propagation

The rate-dependent behavior associated with deformation and fracturing of materials, such as natural rocks, poses significant challenges for modeling. In addition to the complications of the viscoelastic response, the speed of fracture propagation reflects micromechanical mechanisms in the fracture process zone(FPZ). In order to represent these complicated behaviors, a thermodynamically consistent, rate-dependent fracture model is required. Based on rigorous thermodynamic principles, we derive a rate-dependent phase-field mechanical model coupled with single-phase fluid flow in both the matrix and the fracture. The model is guaranteed to satisfy energy conservation during fracture propagation. Here, the system of equations is solved using the introduced solution procedure and a novel preconditioner that accounts for the complex fluid-structure interaction. The proposed phase-field model is tested against several benchmark problems on solid-fluid coupling, fluid-driven fracture propagation and rate-dependent viscoelastic deformation. The model serves as a strong basis for investigating rate-dependent fracturing experiments and for making predictions of material behaviors under new conditions.

42 ENGINEERING↗
DOE OSTI2024

A multiscale anisotropic polymer network model coupled with phase field fracture

The study of polymers has continued to gain substantial attention due to their expanding range of applications, spanning essential engineering fields to emerging domains like stretchable electronics, soft robotics, and implantable sensors. These materials exhibit remarkable properties, primarily stemming from their intricate polymer chain network, which, in turn, increases the complexity of precisely modeling their behavior. Especially for modeling elastomers and their fracture behavior, accurately accounting for the deformations of the polymer chains is vital for predicting the rupture in highly stretched chains. Despite the importance, many robust multiscale continuum frameworks for modeling elastomer fracture tend to simplify network deformations by assuming uniform behavior among chains in all directions. Recognizing this limitation, our study proposes a multiscale fracture model that accounts for the anisotropic nature of elastomer network responses. At the microscale, damage in the chains is assumed to be driven by both the chain's entropy and the internal energy due to molecular bond distortions. In order to bridge the stretching in the chains to the macroscale deformation, we employ the maximal advance path constraint network model, inherently accommodating anisotropic network responses. As a result, chains oriented differently can be predicted to exhibit varying stretch and, consequently, different damage levels. To drive macroscale fracture based on damages in these chains, we utilize the micromorphic regularization theory, which involves the introduction of dual local-global damage variables at the macroscale. The macroscale local damage variable is obtained through the homogenization of the chain damage values, resulting in the prediction of an isotropic material response. The macroscale global damage variable is subjected to nonlocal effects and boundary conditions in a thermodynamically consistent phase field continuum formulation. Moreover, the total dissipation in the system is considered to be mainly due to the breaking of the molecular bonds at the microscale. To validate our model, we employ the double-edge notched tensile test as a benchmark, comparing simulation predictions with existing experimental data. Additionally, to enhance our understanding of the fracturing process, we conduct uniaxial tensile experiments on a square film made up of polydimethylsiloxane (PDMS) rubber embedded with a hole and notches and then compare our simulation predictions with the experimental observations. Furthermore, we visualize the evolution of stretch and damage values in chains oriented along different directions to assess the predictive capacity of the model. In conclusion, the results are also compared with another existing model to evaluate the utility of our model in accurately simulating the fracture behavior of rubber-like materials.

42 ENGINEERING↗
DOE OSTI2021

Second-Order Invariant Domain Preserving ALE Approximation of Euler Equations

Abstract An invariant domain preserving arbitrary Lagrangian-Eulerian method for solving non-linear hyperbolic systems is developed. The numerical scheme is explicit in time and the approximation in space is done with continuous finite elements. The method is made invariant domain preserving for the Euler equations using convex limiting and is tested on various benchmarks.

Guermond, Jean-Luc↗
DOE OSTI2020

Efficient solvers for hybridized three-field mixed finite element coupled poromechanics

We consider a mixed hybrid finite element formulation for coupled poromechanics. A stabilization strategy based on a macro-element approach is advanced to eliminate the spurious pressure modes appearing in undrained/incompressible conditions. The efficient solution of the stabilized mixed hybrid block system is addressed by developing a class of block triangular preconditioners based on a Schur-complement approximation strategy. Robustness, computational efficiency and scalability of the proposed approach are theoretically discussed and tested using challenging benchmark problems on massively parallel architectures.

42 ENGINEERING↗
DOE OSTI2026

Qualifying LaBr3:Ce+Sr detector performance for the Mu2e experiment at Fermilab Using the ELBE Accelerator

A LaBr3:Ce+Sr detector will be used to measure the stopped muon captures at the Mu2e experiment at Fermilab. It has been benchmarked in a test beam experiment performed at the ELBE electron accelerator located at the Helmholtz-Zentrum Dresden-Rossendorf, Germany. ELBE’s pulsed bremsstrahlung beam line was set to deliver an average γ -ray energy of between 4–5 MeV. The detector response was mapped to match Mu2e beam conditions, including rates up to 1 Mcps, energy flux, and time structure. A radioactive calibration source was used to mimic the characteristic 1808.7 keV γ -ray, emitted during the atomic muon nuclear capture in the Mu2e aluminum stopping target. The detector energy resolution was measured as a function of the average energy flux: up to 1 TeV/s for 0.34 s, the steady-operation beam-on time and up to 4 TeV/s for 5 ms to get a conservative estimate of the effect of high intensity Mu2e beam fluctuations. The PMT gain variation as a function of the beam spill length and average intensity has been parametrized and corrected for. When a PMT gain correction corresponding to the average beam-spill intensity is applied, the residual effect of beam intensity fluctuations around the average degrades the energy resolution, σ E γ /E γ , at 1808.7 keV from 0.66% to 0.83%.

Huang, Shihua [Purdue U., West Lafayette]↗
DOE OSTI2025

Exploring nuclear structure with multiparticle azimuthal correlations at the LHC

Details of the nuclear structure of Xe, such as the quadrupole deformation and the nuclear diffuseness, are studied by extensive measurements of anisotropic-flow-related observables in Xe–Xe collisions at a centre-of-mass energy per nucleon pair $\sqrt{s_{NN}}$ = 5.44 TeV with the ALICE detector at the LHC. The results are compared with those from Pb–Pb collisions at $\sqrt{s_{NN}}$ = 5.02 TeV for a baseline, given that the 208 Pb nucleus exhibits a very weak deformation. Furthermore, comprehensive comparisons are performed with a state-of-the-art hybrid model using IP-Glasma+MUSIC+UrQMD. It is found that among various IP-Glasma+MUSIC+UrQMD calculations with different values of nuclear parameters, the one using a nuclear diffuseness parameter of α 0 = 0.492 and a nuclear quadrupole deformation parameter of β 2 = 0.207 provides a better description of the presented flow measurements. These studies represent the first systematic exploration of nuclear structure at TeV energies, utilizing a comprehensive set of anisotropic flow observables. The measurements serve as a critical experimental benchmark for rigorously testing the interplay between nuclear structure inputs and heavy-ion theoretical models.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗