Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58
All sourcesNASA NTRSDOE OSTI
NASA NTRS2009

PrimeSupplier Cross-Program Impact Analysis and Supplier Stability Indicator Simulation Model

PrimeSupplier, a supplier cross-program and element-impact simulation model, with supplier solvency indicator (SSI), has been developed so that the shuttle program can see early indicators of supplier and product line stability, while identifying the various elements and/or programs that have a particular supplier or product designed into the system. The model calculates two categories of benchmarks to determine the SSI, with one category focusing on agency programmatic data and the other focusing on a supplier's financial liquidity. PrimeSupplier was developed to help NASA smoothly transition design, manufacturing, and repair operations from the Shuttle program to the Constellation program, without disruption in the industrial supply base.

Calluzzi, Michael↗
NASA NTRS2009

The Capillary Flow Experiments Aboard the International Space Station: Increments 9-15

This report provides a summary of the experimental, analytical, and numerical results of the Capillary Flow Experiment (CFE) performed aboard the International Space Station (ISS). The experiments were conducted in space beginning with Increment 9 through Increment 16, beginning August 2004 and ending December 2007. Both primary and extra science experiments were conducted during 19 operations performed by 7 astronauts including: M. Fincke, W. McArthur, J. Williams, S. Williams, M. Lopez-Alegria, C. Anderson, and P. Whitson. CFE consists of 6 approximately 1 to 2 kg handheld experiment units designed to investigate a selection of capillary phenomena of fundamental and applied importance, such as large length scale contact line dynamics (CFE-Contact Line), critical wetting in discontinuous structures (CFE-Vane Gap), and capillary flows and passive phase separations in complex containers (CFE-Interior Corner Flow). Highly quantitative video from the simply performed flight experiments provide data helpful in benchmarking numerical methods, confirming theoretical models, and guiding new model development. In an extensive executive summary, a brief history of the experiment is reviewed before introducing the science investigated. A selection of experimental results and comparisons with both analytic and numerical predictions is given. The subsequent chapters provide additional details of the experimental and analytical methods developed and employed. These include current presentations of the state of the data reduction which we anticipate will continue throughout the year and culminate in several more publications. An extensive appendix is used to provide support material such as an experiment history, dissemination items to date (CFE publication, etc.), detailed design drawings, and crew procedures. Despite the simple nature of the experiments and procedures, many of the experimental results may be practically employed to enhance the design of spacecraft engineering systems involving capillary interface dynamics.

Jenson, Ryan M.↗
NASA NTRS2009

Primesupplier Podcast

PrimeSupplier, s supplier cross program and element impact simulation model with supplier solvency indicator (SSI), has been developed so that the Shuttle Program can see an early indicator of supplier and product line stability while identifying the various elements and/or programs that have a particular supplier of product designed into the system. The model calculates two categories of benchmarks to determine the SSI with one category focusing on Agency programmatic and the other being a supplier financial liquidity.

Galluzzi, Michael↗
NASA NTRS2012

The Influence of Pickup Protons, from Interstellar Neutral Hydrogen, on the Propagation of Interplanetary Shocks from the Halloween 2003 Solar Events to ACE and Ulysses: A 3-D MHD Modeling Study

We describe our 3-D, time ]dependent, MHD solar wind model that we recently modified to include the physics of pickup protons from interstellar neutral hydrogen. The model has a time-dependent lower boundary condition, at 0.1 AU, that is driven by source surface map files through an empirical interface module. We describe the empirical interface and its parameter tuning to maximize model agreement with background (quiet) solar wind observations at ACE. We then give results of a simulation study of the famous Halloween 2003 series of solar events. We began with shock inputs from the Fearless Forecast real ]time shock arrival prediction study, and then we iteratively adjusted input shock speeds to obtain agreement between observed and simulated shock arrival times at ACE. We then extended the model grid to 5.5 AU and compared those simulation results with Ulysses observations at 5.2 AU. Next we undertook the more difficult tuning of shock speeds and locations to get matching shock arrival times at both ACE and Ulysses. Then we ran this last case again with neutral hydrogen density set to zero, to identify the effect of pickup ions. We show that the speed of interplanetary shocks propagating from the Sun to Ulysses is reduced by the effects of pickup protons. We plan to make further improvements to the model as we continue our benchmarking process to 10 AU, comparing our results with Cassini observations, and eventually on to 100 AU, comparing our results with Voyager 1 and 2 observations.

Detman, T. R.↗
NASA NTRS2013

Autonomous Aerobraking Development Software: Phase 2 Summary

NASA has used aerobraking at Mars and Venus to reduce the fuel required to deliver a spacecraft into a desired orbit compared to an all-propulsive solution. Although aerobraking reduces the propellant, it does so at the expense of mission duration, large staff, and DSN coverage. These factors make aerobraking a significant cost element in the mission design. By moving on-board the current ground-based tasks of ephemeris determination, atmospheric density estimation, and maneuver sizing and execution, a flight project would realize significant cost savings. The NASA Engineering and Safety Center (NESC) sponsored Phase 1 and 2 of the Autonomous Aerobraking Development Software (AADS) study, which demonstrated the initial feasibility of moving these current ground-based functions to the spacecraft. This paper highlights key state-of-the-art advancements made in the Phase 2 effort to verify that the AADS algorithms are accurate, robust and ready to be considered for application on future missions that utilize aerobraking. The advancements discussed herein include both model updates and simulation and benchmark testing. Rigorous testing using observed flight atmospheres, operational environments and statistical analysis characterized the AADS operability in a perturbed environment.

Cianciolo, Alicia D.↗
DOE OSTI2021

Evaluating Cell Temperature Models and the Effect of Wind Speed in PV System Capacity Testing: Preprint

Capacity testing is a routine procedure for assessing a photovoltaic system's performance relative to expectations. The most common test method involves fitting a regression model that predicts system output power using operating weather conditions including wind speed. Structural modifications to the regression model to incorporate wind in different ways improved the model's ability to fit measured system performance, but the observed improvements were small and unlikely to change the result of a capacity test. However, the results showed that the choice of reporting wind speed and inclusion or exclusion of wind speed in the performance model used as the test benchmark can significantly change the test result.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2020

Benchmark of the LAMMPS code on CRESCO and SUMMIT HPC systems

In this work we present the benchmark results of a numerical model where the solid Deuterium grows under gas pressure. The two-phase molecular Deuterium systems (gas + solid) is modeled using classical molecular dynamics approach and the LAMMPS code. The performance between different modern supercomputers are analyzed and compared. In addition, the numerical simulations reach high accuracy to produce results that match with the data collected through laboratory experiments.

Lupo Pasini, Massimiliano↗
DOE OSTI2023

OECD-NEA HTTF Benchmark Progress and Updates

General slides discussing High Temperature Test Facility benchmark progress, Prismatic HTGR deployment, modeling and simulation tools for validating systems, and verification and validation issues. Also discuss collaboration and conclusions of RELAP5-3D validation activities based on HTTF with trends in data and international impact to accelerate deployment of prismatic HTGR microreactors by providing an opportunity for designers to assess their codes against experimental data and solutions from other codes.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

3D spherical functional expansion tallies in Serpent 2 Monte Carlo code

This work extends the application of functional expansion tallies to 3D spherical geometries. The 3D Zernike polynomials are set as an orthonormal polynomials basis for the functional reconstruction. The study describes the construction of the complete set of polynomials, a natural expansion of the spherical harmonics polynomials where 3D Zernike moments can be evaluated as a linear combination of the geometrical moments. The 3D Zernike polynomials formulation and the computational approach implemented in Serpent 2 are presented and tested through the Godiva model from the ICSBEP criticality benchmark test cases. The implementation results are in agreement with a reference solution described in a fine-resolution mesh, enhancing also the performance and memory demand. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Real-time streamflow forecasting: AI vs. Hydrologic insights

In this paper, we propose a set of simple benchmarks for the evaluation of data-based models for real-time streamflow forecasting, such as those developed with sophisticated Artificial Intelligence (AI) algorithms. The benchmarks are also data-based and provide context to judge incremental improvements in the performance metrics from the more complicated approaches. The benchmarks include temporal and spatial persistence, persistence corrected for baseflow and streamflow, as well as river distance weighted runoff obtained from space-time distributed rainfall. In the development of the benchmarks, we use basic hydrologic insights such as flow aggregation by the river network, scale-dependence in basin response, streamflow partitioning into quick flow and baseflow, water travel time, and rainfall averaging by the basin width function. The study uses 140 streamflow gauges in Iowa that cover a range of basin scales between 7 and 37,000 km 2 . The data cover 17 years. This work demonstrates that the proposed benchmarks can provide good performance according to several commonly used metrics. For example, streamflow forecasting at half of the test locations across years achieves a Kling-Gupta Efficiency (KGE) score of 0.6 or higher at one-day ahead lead time, and 20% of cases reach the KGE of 0.8 or higher. The proposed benchmarks are easy to implement and should prove useful for developers of data-based as well as physics-based hydrologic models and real-time data assimilation techniques.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

On the predictability of turbulent fluxes from land: PLUMBER2 MIP experimental description and preliminary results

Accurate representation of the turbulent exchange of carbon, water, and heat between the land surface and the atmosphere is critical for modelling global energy, water, and carbon cycles in both future climate projections and weather forecasts. Evaluation of models' ability to do this is performed in a wide range of simulation environments, often without explicit consideration of the degree of observational constraint or uncertainty and typically without quantification of benchmark performance expectations. We describe a Model Intercomparison Project (MIP) that attempts to resolve these shortcomings, comparing the surface turbulent heat flux predictions of around 20 different land models provided with in situ meteorological forcing evaluated with measured surface fluxes using quality-controlled data from 170 eddy-covariance-based flux tower sites. Predictions from seven out-of-sample empirical models are used to quantify the information available to land models in their forcing data and so the potential for land model performance improvement. Sites with unusual behaviour, complicated processes, poor data quality, or uncommon flux magnitude are more difficult to predict for both mechanistic and empirical models, providing a means of fairer assessment of land model performance. When examining observational uncertainty, model performance does not appear to improve in low-turbulence periods or with energy-balance-corrected flux tower data, and indeed some results raise questions about whether the energy balance correction process itself is appropriate. In all cases the results are broadly consistent, with simple out-of-sample empirical models, including linear regression, comfortably outperforming mechanistic land models. In all but two cases, latent heat flux and net ecosystem exchange of CO 2 are better predicted by land models than sensible heat flux, despite it seeming to have fewer physical controlling processes. Land models that are implemented in Earth system models also appear to perform notably better than stand-alone ecosystem (including demographic) models, at least in terms of the fluxes examined here. The approach we outline enables isolation of the locations and conditions under which model developers can know that a land model can improve, allowing information pathways and discrete parameterisations in models to be identified and targeted for future model development.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Emerging Flexible Designs for Geospatial Multimodal Foundation Models

Foundation models are rapidly transforming Earth observation by enabling scalable pretraining across diverse unlabeled geospatial modalities. However, their architectural diversity—ranging from encoder-only to encoder-decoder and masked autoencoding paradigms—makes it challenging to assess performance trade-offs in a consistent manner. In this work, we present an apples-to-apples comparison of leading FM architectures designed for geospatial multimodal reasoning, with a particular focus on flexibility across varied spectral band configurations. We standardize pretraining using identical self-supervised learning objectives and training datasets, and evaluate all models under consistent parameterization on the GEOBench benchmark across classification and segmentation tasks. Our results offer new insights into the design trade-offs between model flexibility, modality alignment, and downstream task performance. By highlighting architectural strengths and limitations under controlled conditions, this study provides practical guidance for building next-generation geospatial foundation models capable of robust multimodal reasoning.

Ambrozio Dias, Philipe [ORNL] (ORCID:0000000194277↗
DOE OSTI2025

Warm-phase microphysical evolution in large-eddy simulations of tropical cumulus congestus: evaluating drop size distribution evolution using polarimetry retrievals, in situ measurements, and a thermal-based framework

Owing to uncertainties in convective microphysics processes, improving parameterizations in Earth system models (ESMs) can benefit from observationally constrained cases suitable for scaling between cloud-resolving models and ESMs. We propose a benchmark large-eddy simulation (LES) cumulus congestus case study from the NASA Cloud, Aerosol, and Monsoon Processes Philippines Experiment (CAMP 2 Ex) for evaluating and improving ESMs in single-column model (SCM) mode. We seek observational constraints using novel polarimetric retrievals and in situ cloud microphysics measurements. Simulations using bulk and bin microphysics initialized with observed aerosol profiles are compared to cloud-top retrievals of cloud droplet effective radius (R eff ), effective variance (ν eff ), and number concentration (N d ) from the airborne Research Scanning Polarimeter (RSP). Both schemes reproduce characteristics of cloud-top N d and R eff that increase and decrease with altitude, respectively. Cloud-top N d is low-biased relative to RSP retrievals in both schemes, potentially due to limitations in both simulations and retrieval assumptions. Cloud-top R eff is low-biased in the bulk scheme but reasonably reproduced by the bin scheme. Profiles of N d and R eff are sensitive to the collision–coalescence process and the vertical variation in aerosol size distribution. Comparison of simulated and in situ droplet size distributions (DSDs) shows that, to first order, integrated moments are always sensitive to sizes < ~ 30 µm and can also be sensitive to larger sizes if the DSDs are sufficiently broad, with implications for the assumed maximum observed size retrieved by the RSP. The bin scheme captures the observed extended tail of the DSD, while the bulk scheme is unable to due to parametric constraints. Differences in expected relationships between in situ measurements of cloud cores and cloud-top retrievals by RSP demonstrate difficulty in constraining well the case presented herein. Finally, a thermal-tracking framework demonstrates that the dilution of N d throughout a thermal's lifetime is heavily determined by collision–coalescence and the height-varying aerosol distribution and that, in the absence of these, the impact of entrainment on diluting N d is largely offset by secondary activation. Implications for evaluating warm-phase convective microphysics schemes in ESMs and translating results for use on global, space-based polarimetry platforms are discussed.

Stanford, McKenna Wallace [Columbia Univ., New Yor↗
DOE OSTI2023

Federated benchmarking of medical artificial intelligence with MedPerf

Medical artificial intelligence (AI) has tremendous potential to advance healthcare by supporting and contributing to the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving both healthcare provider and patient experience. Unlocking this potential requires systematic, quantitative evaluation of the performance of medical AI models on large-scale, heterogeneous data capturing diverse patient populations. Here, to meet this need, we introduce MedPerf, an open platform for benchmarking AI models in the medical domain. MedPerf focuses on enabling federated evaluation of AI models, by securely distributing them to different facilities, such as healthcare organizations. This process of bringing the model to the data empowers each facility to assess and verify the performance of AI models in an efficient and human-supervised process, while prioritizing privacy. We describe the current challenges healthcare and AI communities face, the need for an open platform, the design philosophy of MedPerf, its current implementation status and real-world deployment, our roadmap and, importantly, the use of MedPerf with multiple international institutions within cloud-based technology and on-premises scenarios. Finally, we welcome new contributions by researchers and organizations to further strengthen MedPerf as an open benchmarking platform.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Code Benchmark of the HTTF Pressurized Conduction Cooldown Test Using SAM

The High Temperature Test Facility (HTTF) at Oregon State University is an integral system test facility to simulate postulated reactor transients of prismatic high-temperature gas-cooled reactors(HTGRs). A series of test campaigns was launched, providing abundant test data that could be used to benchmark reactor system analysis codes like the System Analysis Module (SAM). In this study, a SAM model of the facility is developed based on the two-dimensional (2D) ring model approach. All components including the ceramic matrix, graphite heaters, helium coolant channels, core barrel, upcomer, pressure vessel, and reactor cavity cooling system are modeled as concentric cylindrical rings. The model is used to simulate one of the benchmark problems-Pressurized Conduction Cooldown (PCC)-within the scope of the Organisation for Economic Co-operation and Development Nuclear Energy Agency International HTTF Benchmark. The simulations consist of two parts. In the first part, operating and boundary conditions as well as thermophysical properties of materials are specified for the benchmark problem. In this work, results from the first part will be used in code-to-code comparison. In the second part, the SAM model is used to simulate Test PG-27, which is the first PCC test carried out in the HTTF, with only two of the ten heater banks activated. The results in the second part are used for code-to-data comparison. Because the helium coolant flow rate is not measured in this facility, it is estimated using the input power and inlet/outlet coolant temperatures. Additionally, radial heat flow in the ceramic blocks is complicated by hundreds of cylindrical coolant channels and heater rods embedded in them. As such, it is necessary to deduce an effective thermal conductivity for the ceramic to analyze the core thermal behavior. SAM predictions of the helium coolant and ceramic temperatures are compared with test data measured in three equivalent sectors. Overall, the SAM results agree reasonably well with test data within the variation of data among the three sectors, which demonstrates SAM's capability in capturing transient effects in HTGR using the simplified 2D ring model.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

RELAP5-3D HTGR Validation Work at Idaho National Laboratory

Prismatic block-type high-temperature gas-cooled reactors (HTGRs) were built in the United States decades ago, and now advanced reactor vendors are seeking to deploy them again for a variety of applications. Deploying these reactors requires modelling and simulation tools that have been validated against conditions representative of the HTGR application. Idaho National Laboratory (INL) is leading the execution an of HTGR thermal hydraulics benchmark to accelerate the validation of thermal hydraulics modelling and simulation tools for these applications. That benchmark is based on a facility called the High Temperature Test Facility (HTTF). This work provides an overview of work conducted at INL over the last 2 years to validate RELAP5-3D against data from HTTF. This presentation shows results from multiple RELAP5-3D models and an HTTF experiment to assess the impact of certain modelling assumptions on results. The contents of this talk sit on the cutting edge of RELAP5-3D validation for HTGR analysis.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

A smeared crack modeling framework accommodating multi-directional fracture at finite strains

A generic smeared crack modeling framework predicated on the deformation gradient decomposition (DGD) approach is proposed for use in dynamic fracture problems at finite strains, accommodating failure along multiple mutually orthogonal fracture planes embedded within an independently defined bulk material model. Within this constitutive framework, the traction equilibrium conditions imposed at each failure surface are used to determine the associated crack displacements stored as internal state variables. In general, the enforcement of interfacial equilibrium entails the implicit solution of a non-linear system of equations within the constitutive update procedure. However, if inertial effects arising due to the relative motion of the fractured material are incorporated within the model, the traction equilibrium conditions are shown to give rise to corresponding dynamic equations of motion governing the time-evolution of the crack opening displacements. For dynamic problems, an explicit time-integration procedure is devised to efficiently update the material state, subject to a set of internal frictionless contact constraints to prevent material inversion. Finally, the efficacy of the proposed modeling framework is investigated through several benchmark dynamic fracture problems run within the explicit finite element code DYNA3D.

42 ENGINEERING↗
NASA NTRS1987

Finite element analysis of steady and transiently moving/rolling nonlinear viscoelastic structure. II - Shell and three-dimensional simulations

In a three-part series of papers, a generalized finite element solution strategy is developed to handle traveling load problems in rolling, moving and rotating structure. The main thrust of this section consists of the development of three-dimensional and shell type moving elements. In conjunction with this work, a compatible three-dimensional contact strategy is also developed. Based on these modeling capabilities, extensive analytical and experimental benchmarking is presented. Such testing includes traveling loads in rotating structure as well as low- and high-speed rolling contact involving standing wave-type response behavior. These point to the excellent modeling capabilities of moving element strategies.

Kennedy, Ronald↗