Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,423 records · Page 79

Benchmarking State-of-the-Art Theory and Empirical Models of Pionless Neutrino-Argon Scattering in GENIE

Neutrino event generators require a delicate balance between theory and empirically-driven models to achieve reliable simulations. GENIE is the most commonly used generator, bridging theory and experiment in modern neutrino physics. Its flexible framework makes it ideal for comparing different models across all aspects of neutrino interactions—from the nuclear ground state and primary vertex to final-state interactions. Recently, GENIE has incorporated several state-of-the-art, theory-driven models, including spectral-function descriptions of the nuclear ground state, axial form factors from first-principles lattice QCD calculations, and the Liège intranuclear cascade model for simulating final-state interactions. Compared to the previous baseline models in GENIE—such as the local Fermi gas, dipole-like axial form factor, and hA final-state interaction model—the new implementations are more physically realistic and incorporate more complete physics, including nuclear de-excitation. This poster presents the implementation of new models in GENIE and their performance in comparisons among these models against recent MicroBooNE cross-section measurements.

Liu, Liang [Fermilab] (ORCID:000000026753925X)↗

Benchmarking State-of-the-Art Theory and Empirical Models of Pionless Neutrino-Argon Scattering in GENIE

Neutrino event generators require a delicate balance between theory and empirically-driven models to achieve reliable simulations. GENIE is the most commonly used generator, bridging theory and experiment in modern neutrino physics. Its flexible framework makes it ideal for comparing different models across all aspects of neutrino interactions—from the nuclear ground state and primary vertex to final-state interactions. Recently, GENIE has incorporated several state-of-the-art, theory-driven models, including spectral-function descriptions of the nuclear ground state, axial form factors from first-principles lattice QCD calculations, and the Liège intranuclear cascade model for simulating final-state interactions. Compared to the previous baseline models in GENIE—such as the local Fermi gas, dipole-like axial form factor, and hA final-state interaction model—the new implementations are more physically realistic and incorporate more complete physics, including nuclear de-excitation. This poster presents the implementation of new models in GENIE and their performance in comparisons among these models against recent MicroBooNE cross-section measurements.

Liu, Liang [Fermilab] (ORCID:000000026753925X)↗

An alternative assessment of second-order closure models in turbulent shear flows

The performance of three recently proposed second-order closure models is tested in benchmark turbulent shear flows. Both homogeneous shear flow and the log-layer of an equilibrium turbulent boundary layer are considered for this purpose. An objective analysis of the results leads to an assessment of these models that stands in contrast to that recently published by other authors. A variety of pitfalls in the formulation and testing of second-order closure models are uncovered by this analysis.

Speziale, Charles G.↗

An Experimental Study of Characteristic Combustion-Driven Flow for CFD Validation

A series of uni-element rocket injector studies were completed to provide benchmark quality data needed to validate computational fluid dynamic models. A shear coaxial injector geometry was selected as the primary injector for study using gaseous hydrogen/oxygen and gaseous hydrogen/liquid oxygen propellants. Emphasis was placed on the use of nonintrusive diagnostic techniques to characterize the flowfields inside an optically-accessible rocket chamber. Measurements of the velocity and species fields were obtained using laser velocimetry and Raman spectroscopy, respectively. Qualitative flame shape information was also obtained using laser-induced fluorescence excited from OH radicals and laser light scattering studies of aluminum oxide particle seeded combusting flows. The gaseous hydrogen/liquid oxygen propellant studies for the shear coaxial injector focused on breakup mechanisms associated with the liquid oxygen jet under subcritical pressure conditions. Laser sheet illumination techniques were used to visualize the core region of the jet and a Phase Doppler Particle Analyzer was utilized for drop velocity, size and size distribution characterization. The results of these studies indicated that the shear coaxial geometry configuration was a relatively poor injector in terms of mixing. The oxygen core was observed to extend well downstream of the injector and a significant fraction of the mixing occurred in the near nozzle region where measurements were not possible to obtain. Detailed velocity and species measurements were obtained to allow CFD model validation and this set of benchmark data represents the most comprehensive data set available to date. As an extension of the investigation, a series of gas/gas injector studies were conducted in support of the X-33 Reusable Launch Vehicle program. A Gas/Gas Injector Technology team was formed consisting of the Marshall Space Flight Center, the NASA Lewis Research Center, Rocketdyne and Penn State. Injector geometries studied under this task included shear and swirl coaxial configurations as well as an impinging jet injector.

Santoro, Robert J.↗

An Experimental Study of Characteristic Combustion-Driven Flow for CFD Validation

A series of uni-element rocket injector studies were completed to provide benchmark quality data needed to validate computational fluid dynamic models. A shear coaxial injector geometry was selected as the primary injector for study using gaseous hydrogen/oxygen and gaseous hydrogen/liquid oxygen propellants. Emphasis was placed on the use of non-intrusive diagnostic techniques to characterize the flowfields inside an optically-accessible rocket chamber. Measurements of the velocity and species fields were obtained using laser velocimetry and Raman spectroscopy, respectively Qualitative flame shape information was also obtained using laser-induced fluorescence excited from OH radicals and laser light scattering studies of aluminum oxide particle seeded combusting flows. The gaseous hydrogen/liquid oxygen propellant studies for the shear coaxial injector focused on breakup mechanisms associated with the liquid oxygen jet under sub-critical pressure conditions. Laser sheet illumination techniques were used to visualize the core region of the jet and a Phase Doppler Particle Analyzer was utilized for drop velocity, size and size distribution characterization. The results of these studies indicated that the shear coaxial geometry configuration was a relatively poor injector in terms of mixing. The oxygen core was observed to extend well downstream of the injector and a significant fraction of the mixing occurred in the near nozzle region where measurements were not possible to obtain Detailed velocity and species measurements were obtained to allow CFD model validation and this set of benchmark data represents the most comprehensive data set available to date As an extension of the investigation, a series of gas/gas injector studies were conducted in support of the X-33 Reusable Launch Vehicle program. A Gas/Gas Injector Technology team was formed consisting of the Marshall Space Flight Center, the NASA Lewis Research Center, Rocketdyne and Penn State. Injector geometries studied under this task included shear and swirl coaxial configurations as well as an impinging jet injector.

Santoro, Robert J.↗

Analysis of Ice Mass Growth Over Time on the CRM65 Midspan Hybrid Model

The Aeronautics Research Mission Directorate at NASA is developing and applying tools to enable future technologies towards sustainable flight. Aircraft icing has been identified as a potential barrier to entry into service for innovative designs necessitating improvements to computational ice accretion tools. NASA is developing the Glenn Icing Computational Environment (GlennICE) to address deficiencies in the computational modeling capabilities of previously developed ice accretion solvers. To benchmark and improve the ability to model highly three-dimensional ice accretion, high quality validation data against experimental data is required. The CRM65 Midspan Hybrid geometry was previously tested at the NASA Icing Research Tunnel to generate experimental data for swept wing geometries typical for commercial transport aircraft. As a part of a 2018 icing test campaign, experimental data characterizing the relationship between ice accretion time and ice mass growth was obtained and can be leveraged for use in validation of computational tools. The desire for computational ice accretion solvers to predict ice shapes profiles accreted experimentally has often overshadowed the comparison to the mass and bulk volume of ice accreted. To address this deficiency, an analysis is presented in which GlennICE is applied to simulations of the CRM65 Midspan Hybrid model tested in the NASA Icing Research Tunnel. Results from the computational fluid dynamics simulations compared favorably to the experimental pressure coefficient data, thus validating the modeling setup. The experimental data showed excellent repeatability for the 15.0 minute accretion time. The comparisons between the experimental and computational ice mass over time showed good agreement up to 10.0 minutes after which the ice mass was underpredicted. The experimental ice mass was largely linear with some nonlinear data. The bulk volume of ice accreted experimentally compared well to GlennICE for the scanned ice shapes and mean combined cross section ice shapes, but was underpredicted for the maximum combined cross section ice shapes at longer accretion times. The experimental minimum combined cross section, mean combined cross section, and maximum combined cross section profiles when compared to GlennICE show good agreement for the mean combined cross section up to 15.0 minutes. The analyses show that with a single-shot method, GlennICE currently underpredicts the ice mass for longer accretion times, is not able to match the bulk volume of the maximum combined cross section due to dominating scallop features, and future work is required to generate a more generalized ice bulk density model.

Icing↗

Analysis of Ice Mass Growth Over Time on the CRM65 Midspan Hybrid Model

The Aeronautics Research Mission Directorate at NASA is developing and applying tools to enable future technologies towards sustainable flight. Aircraft icing has been identified as a potential barrier to entry into service for innovative designs necessitating improvements to computational ice accretion tools. NASA is developing the Glenn Icing Computational Environment (GlennICE) to address deficiencies in the computational modeling capabilities of previously developed ice accretion solvers. To benchmark and improve the ability to model highly three-dimensional ice accretion, high quality validation data against experimental data is required. The CRM65 Midspan Hybrid geometry was previously tested at the NASA Icing Research Tunnel to generate experimental data for swept wing geometries typical for commercial transport aircraft. As a part of a 2018 icing test campaign, experimental data characterizing the relationship between ice accretion time and ice mass growth was obtained and can be leveraged for use in validation of computational tools. The desire for computational ice accretion solvers to predict ice shapes profiles accreted experimentally has often overshadowed the comparison to the mass and bulk volume of ice accreted. To address this deficiency, an analysis is presented in which GlennICE is applied to simulations of the CRM65 Midspan Hybrid model tested in the NASA Icing Research Tunnel. Results from the computational fluid dynamics simulations compared favorably to the experimental pressure coefficient data, thus validating the modeling setup. The experimental data showed excellent repeatability for the 15.0 minute accretion time. The comparisons between the experimental and computational ice mass over time showed good agreement up to 10.0 minutes after which the ice mass was underpredicted. The experimental ice mass was largely linear with some nonlinear data. The bulk volume of ice accreted experimentally compared well to GlennICE for the scanned ice shapes and mean combined cross section ice shapes, but was underpredicted for the maximum combined cross section ice shapes at longer accretion times. The experimental minimum combined cross section, mean combined cross section, and maximum combined cross section profiles when compared to GlennICE show good agreement for the mean combined cross section up to 15.0 minutes. The analyses show that with a single-shot method, GlennICE currently underpredicts the ice mass for longer accretion times, is not able to match the bulk volume of the maximum combined cross section due to dominating scallop features, and future work is required to generate a more generalized ice bulk density model.

Icing↗

High-Temperature Gas-Cooled Pebble-Bed Reactors Running In And Transient Modeling Capabilities Demonstration

This study presents a comprehensive benchmarking and verification effort of several thermal-hydraulic and multiphysics capabilities for high-temperature gas-cooled reactor (HTGR) applications. The first part of this effort focuses on the running-in verification of Griffin's multiphysics capabilities, specifically for simulating the evolution of Pebble Bed reactor cores from startup to equilibrium. In the absence of validation data, code-to-code comparisons are conducted with Kugelpy, showing good agreement for key quantities like maximum power density and fresh core k-eigenvalue predictions. However, discrepancies in equilibrium core predictions suggest potential issues with cross sections, underscoring the need for further refinement and evaluation. The HTTF system analysis code benchmark involves RELAP5-3D, SAM, and GAMMA+ to assess their predictive capabilities for HTTF behavior under both normal operation and pressurized conduction cooldown (PCC) transient conditions. While there is good agreement in predicting major parameters such as coolant temperature, solid temperature, and flow distribution, discrepancies in transient behavior highlight differences in modeling approaches, nodalizations, and heat transfer models. The HTTF lower plenum CFD benchmark employs nekRS to simulate flow mixing phenomena, successfully capturing relevant flow physics and demonstrating mesh independence in complex geometries. Preliminary results suggest a relatively uniform temperature field but significant unsteadiness in the flow, requiring time-averaging analyses. The GPBR200 system analysis code benchmark uses SAM's core channel and porous media models, incorporating an RCCS loop for decay heat removal. During steady-state and transient conditions, including protected de-pressurized and pressurized loss of forced cooling (DLOFC and PLOFC), both models show good agreement in predicting temperature profiles and key parameters. Notably, while the core channel model underpredicts convective heat transfer effects, both models maintain temperatures well below the TRISO fuel safety limit. These benchmarking efforts collectively enhance the predictive capabilities of the tools used in HTGR design and safety analysis, guiding developments to improve their accuracy and applicability.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Self-consistent continuum solvation for optical absorption of complex molecular systems in solution

We introduce a new method to compute the optical absorption spectra of complex molecular systems in solution, based on the Liouville approach to time-dependent density-functional perturbation theory and the revised self-consistent continuum solvation model. The former allows one to obtain the absorption spectrum over a whole wide frequency range, using a recently proposed Lanczos-based technique, or selected excitation energies, using the Casida equation, without having to ever compute any unoccupied molecular orbitals. The latter is conceptually similar to the polarizable continuum model and offers the further advantages of allowing an easy computation of atomic forces via the Hellmann-Feynman theorem and a ready implementation in periodic-boundary conditions. The new method has been implemented using pseudopotentials and plane-wave basis sets, benchmarked against polarizable continuum model calculations on 4-aminophthalimide, alizarin, and cyanin and made available through the QUANTUM ESPRESSO distribution of open-source codes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Benchmark of interfacial area concentration approaches for the two-fluid model in gas-dispersed condensing flow

Accurate predictions of interfacial area concentration (IAC) are required for closure of the condensation phase change term. The interfacial area transport equation (IATE) can dynamically predict interfacial area concentration and is increasingly being used in the benchmarking of phase change flows over static based correlations which require a flow regime map. For higher flow regimes, the two-group two-fluid IATE model can separately model the transport of spherical bubbles and cap/slug bubbles. Recently, a modified formulation of the one-group and two-group two-fluid model with interfacial area transport equation was proposed for gas-dispersed condensing flows. Three datasets have been used for validation consisting of condensing two-phase flow data in the unheated section of a vertical annulus with conditions spanning a range of pressures, inlet subcoolings and mass fluxes for benchmarking the various IAC approaches. For bubbly flows, the coupled void transport - IATE simulation results in good agreement in the predictions of the void fraction and the interfacial area concentration under nearly all sixty conditions. Considering the proposed two-group two-fluid IATE model formulation, the group-2 Nusselt number correlation is investigated to account for the appropriate heat transfer length scale for group-2 bubbles. Validation of the proposed two-group two-fluid model against the new dataset shows good agreement with experimental data with significant improvement over the one-group IATE model for conditions with substantial group-2 void fraction throughout the simulation domain.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Advancing the Predictability of Water Cycle Phenomena via the Application of AI to Model Ensemble Simulations and Observations

AI has the power to identify new pathways to extended predictability of the water cycle via its application to model ensembles together with accompanying observations. We discuss both scientific and technological aspects of this challenge, and address the parts of the MODEX approach involving “Model simulations, evaluation, analysis, and benchmarking” and “Identification of key knowledge gaps”. The widespread incorporation of AI into model analysis will significantly advance Earth system predictability and our predictive capabilities.

54 ENVIRONMENTAL SCIENCES↗

Development of Benchmark Examples for Delamination Onset and Fatigue Growth Prediction

An approach for assessing the delamination propagation and growth capabilities in commercial finite element codes was developed and demonstrated for the Virtual Crack Closure Technique (VCCT) implementations in ABAQUS. The Double Cantilever Beam (DCB) specimen was chosen as an example. First, benchmark results to assess delamination propagation capabilities under static loading were created using models simulating specimens with different delamination lengths. For each delamination length modeled, the load and displacement at the load point were monitored. The mixed-mode strain energy release rate components were calculated along the delamination front across the width of the specimen. A failure index was calculated by correlating the results with the mixed-mode failure criterion of the graphite/epoxy material. The calculated critical loads and critical displacements for delamination onset for each delamination length modeled were used as a benchmark. The load/displacement relationship computed during automatic propagation should closely match the benchmark case. Second, starting from an initially straight front, the delamination was allowed to propagate based on the algorithms implemented in the commercial finite element software. The load-displacement relationship obtained from the propagation analysis results and the benchmark results were compared. Good agreements could be achieved by selecting the appropriate input parameters, which were determined in an iterative procedure.

Krueger, Ronald↗

Benchmark calculations of radiative forcing by greenhouse gases

Changes in concentrations of greenhouse gases lead to changes in radiative fluxes throughout the atmosphere. The value of this change, the instantaneous radiative forcing, varies across climate models, due partly to differences in the distribution of clouds, humidity, and temperature across models, and partly due to errors introduced by approximate treatments of radiative transfer. This paper describes an experiment within the Radiative Forcing Model Intercomparision Project that uses benchmark calculations made with line–by–line models to identify parameterization error in the representation of absorption and emission by greenhouse gases. Clear–sky instantaneous forcing by greenhouse gases is computed using a set of 100 profiles, selected from a re–analysis of present–day conditions, that represent the global annual mean forcing from pre–industrial times to the present–day with sampling errors of less than 0.01 W m –2 . Six contributing line–by–line models agree in their estimate of this forcing to within 0.025 W m –2 while even recently–developed parameterizations have typical errors four or more times larger, suggesting both that the samples reveal true differences among line–by–line models and that parameterization error will be readily identifiable. Agreement among line–by–line models is better in the longwave than in the shortwave where differing treatments of the water vapor vapor continuum affect estimates of forcing by carbon dioxide and methane. The impacts of clouds on instantaneous radiative forcing are estimated from climate model simulations. Here, the adjustment due to stratospheric temperature change by assuming fixed dynamical heating. Adjustments are large only for ozone and for carbon dioxide, for which stratospheric cooling introduces modest non–linearity.

54 ENVIRONMENTAL SCIENCES↗

MArVD2: a machine learning enhanced tool to discriminate between archaeal and bacterial viruses in viral datasets

Abstract Our knowledge of viral sequence space has exploded with advancing sequencing technologies and large-scale sampling and analytical efforts. Though archaea are important and abundant prokaryotes in many systems, our knowledge of archaeal viruses outside of extreme environments is limited. This largely stems from the lack of a robust, high-throughput, and systematic way to distinguish between bacterial and archaeal viruses in datasets of curated viruses. Here we upgrade our prior text-based tool (MArVD) via training and testing a random forest machine learning algorithm against a newly curated dataset of archaeal viruses. After optimization, MArVD2 presented a significant improvement over its predecessor in terms of scalability, usability, and flexibility, and will allow user-defined custom training datasets as archaeal virus discovery progresses. Benchmarking showed that a model trained with viral sequences from the hypersaline, marine, and hot spring environments correctly classified 85% of the archaeal viruses with a false detection rate below 2% using a random forest prediction threshold of 80% in a separate benchmarking dataset from the same habitats.

Vik, Dean (ORCID:000000027546899X)↗

THE METALLICITY OF THE CM DRACONIS SYSTEM

The CM Draconis system comprises two eclipsing mid-M dwarfs of nearly equal mass in a 1.27 day orbit. This well-studied eclipsing binary has often been used for benchmark tests of stellar models, since its components are among the lowest mass stars with well-measured masses and radii ({approx}< 1% relative precision). However, as with many other low-mass stars, non-magnetic models have been unable to match the observed radii and effective temperatures for CM Dra at the 5%-10% level. To date, the uncertain metallicity of the system has complicated comparison of theoretical isochrones with observations. In this Letter, we use data from the SpeX instrument on the NASA Infrared Telescope Facility to measure the metallicity of the system during primary and secondary eclipses, as well as out of eclipse, based on an empirical metallicity calibration in the H and K near-infrared (NIR) bands. We derive an [Fe/H] = -0.30 {+-} 0.12 that is consistent across all orbital phases. The determination of [Fe/H] for this system constrains a key dimension of parameter space when attempting to reconcile model isochrone predictions and observations.

79 ASTRONOMY AND ASTROPHYSICS↗

Measurement of Close-in Ground Motion from an Underground Chemical Explosion

Understanding the geophysical response near an underground explosion is crucial for generating insights into the source and emplacement conditions that produce distinct observations in monitoring scenarios occurring at greater distances. Recently, Shot A of the Low Yield Nuclear Monitoring (LYNM) Physics Experiment 1 (PE1) series was conducted at the Nevada National Security Site to provide ground truth for subsurface explosion signal models. This experiment resulted in measuring near-source ground motion at distances ranging from 70 to 1000 m/kt with a 99% success rate, yielding high-fidelity knowledge of the near-field response that can serve as benchmarks for future numerical modeling and experiment planning. However, technical challenges exist in observing near-source phenomena while safeguarding sensitive data acquisition components from the detrimental effects of ground motion in the subsurface. This report outlines tools and techniques to address challenges associated with observing near-source accelerations and within the tunnel drift of the PE1 test bed. Additionally, we describe key systems designed with both modern advancements and legacy guidance to maximize the collection of high-quality ground motion data, which may be applied to constitutive and computational models, leading to new or improved understanding of the near- and far-field signals produced by underground explosions.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Are Terrestrial Biosphere Models Fit for Simulating the Global Land Carbon Sink?

The Global Carbon Project estimates that the terrestrial biosphere has absorbed about one-third of anthropogenic CO 2 emissions during the 1959–2019 period. This sink-estimate is produced by an ensemble of terrestrial biosphere models and is consistent with the land uptake inferred from the residual of emissions and ocean uptake. The purpose of our study is to understand how well terrestrial biosphere models reproduce the processes that drive the terrestrial carbon sink. One challenge is to decide what level of agreement between model output and observation-based reference data is adequate considering that reference data are prone to uncertainties. To define such a level of agreement, we compute benchmark scores that quantify the similarity between independently derived reference data sets using multiple statistical metrics. Models are considered to perform well if their model scores reach benchmark scores. Our results show that reference data can differ considerably, causing benchmark scores to be low. Model scores are often of similar magnitude as benchmark scores, implying that model performance is reasonable given how different reference data are. While model performance is encouraging, ample potential for improvements remains, including a reduction in a positive leaf area index bias, improved representations of processes that govern soil organic carbon in high latitudes, and an assessment of causes that drive the inter-model spread of gross primary productivity in boreal regions and humid tropics. The success of future model development will increasingly depend on our capacity to reduce and account for observational uncertainties.

58 GEOSCIENCES↗

Energy and Performance Benchmarking of a Domain Wall-Magnetic Tunnel Junction Multibit Adder

The domain wall (DW)-magnetic tunnel junction (MTJ) device implements universal Boolean logic in a manner that is naturally compact and cascadable. However, an evaluation of the energy efficiency of this emerging technology for standard logic applications is still lacking. In this work, we use a previously developed compact model to construct and benchmark a 32-bit adder entirely from DW-MTJ devices that communicates with DW-MTJ registers. The results of this large-scale design and simulation indicate that, while the energy cost of systems driven by spin-transfer torque (STT) domain wall motion is significantly higher than previously predicted, the same concept using spin-orbit torque (SOT) switching benefits from an improvement in the energy per operation by multiple orders of magnitude, attaining competitive energy values relative to a comparable CMOS sub-processor component. This result clarifies the path towards practical implementations of an all-magnetic processor system.

42 ENGINEERING↗