Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

U.S. Solar Photovoltaic System and Energy Storage Cost Benchmarks, With Minimum Sustainable Price Analysis: Q1 2023

The U.S. Department of Energy's (DOE's) Solar Energy Technologies Office (SETO) aims to accelerate the advancement and deployment of solar technology in support of an equitable transition to a decarbonized economy no later than 2050, starting with a decarbonized power sector by 2035. Its approach to achieving this goal includes driving innovations in technology, hardware, and soft cost reductions to make solar affordable and accessible for all. As part of this effort, SETO must track solar cost trends so it can focus its research and development (R&D) on the highest-impact activities. The benchmarks in this report are bottom-up cost estimates of all major inputs to PV and energy storage system installations. Bottom-up costs are based on national averages and do not necessarily represent typical costs in all local markets. Like last year's report, this year's report includes two distinct sets of benchmarks: minimum sustainable price (MSP) benchmarks and modeled market price (MMP) benchmarks. MSP benchmarks can be interpreted as the minimum price a company needs to charge to remain financially solvent in the long term based on the minimum sustainable prices of all inputs including minimum sustainable profit margins. MMP benchmarks can be interpreted as the actual cash sales price a company charges in the given benchmark period. These simplified estimates are useful for tracking technological progress, but they do not reflect all experiences. In fact, no individual estimate under any approach can reflect the diversity of the PV and storage manufacturing and installation industries. Our residential MMP benchmark ($2.90 per watt direct current [Wdc]) is 24% higher than the MSP benchmark ($2.34/Wdc) and 9% lower than our MMP benchmark ($3.18/Wdc) from Q1 2022 in 2022 U.S. dollars (USD). For community solar, our MMP benchmark ($1.75/Wdc) is 18% higher than our MSP benchmark ($1.49/Wdc). Our Q1 2022 benchmark report has no community solar system for comparison. For utility-scale systems with one-axis tracking, our MMP benchmark ($1.17/Wdc) is 22% higher than our MSP benchmark ($0.96/Wdc) and 10% higher than its counterpart ($1.07/Wdc) in Q1 2022 in 2022 USD.

14 SOLAR ENERGY↗

Q1 2023 U.S. Solar Photovoltaic System and Energy Storage Cost Benchmarks With Minimum Sustainable Price Analysis Data File

The U.S. Department of Energy's (DOE's) Solar Energy Technologies Office (SETO) aims to accelerate the advancement and deployment of solar technology in support of an equitable transition to a decarbonized economy no later than 2050, starting with a decarbonized power sector by 2035. Its approach to achieving this goal includes driving innovations in technology, hardware, and soft cost reductions to make solar affordable and accessible for all. As part of this effort, SETO must track solar cost trends so it can focus its research and development (R&D) on the highest-impact activities. The benchmarks in this report are bottom-up cost estimates of all major inputs to PV and energy storage system installations. Bottom-up costs are based on national averages and do not necessarily represent typical costs in all local markets. Like last year's report, this year's report includes two distinct sets of benchmarks: minimum sustainable price (MSP) benchmarks and modeled market price (MMP) benchmarks. MSP benchmarks can be interpreted as the minimum price a company needs to charge to remain financially solvent in the long term based on the minimum sustainable prices of all inputs including minimum sustainable profit margins. MMP benchmarks can be interpreted as the actual cash sales price a company charges in the given benchmark period. These simplified estimates are useful for tracking technological progress, but they do not reflect all experiences. In fact, no individual estimate under any approach can reflect the diversity of the PV and storage manufacturing and installation industries.

14 SOLAR ENERGY↗

MODELING AND SIMULATION OF SOLID FLUIDIZATION IN A RESIN COLUMN

The objective of the present work is to model the resin particles within the column during fluidization and sedimentation processes using computation fluid dynamics (CFD) approach. The calculated results will help interpret experimental results, and they will assist in providing guidance on specific details of testing design and establishing a basic understanding of particle’s hydraulic characteristics within the column. The model is benchmarked against the literature data and the test data (2003) conducted at Savannah River Site (SRS). The paper presents the benchmarking results and the modeling predictions of the SRS resin column using the improved literature correlations applicable for liquid-solid granular flow.

42 ENGINEERING↗

Radiation Detection Computational Benchmark Scenarios

Modeling forms an important component of radiation detection development, allowing for testing of new detector designs, evaluation of existing equipment against a wide variety of potential threat sources, and assessing operation performance of radiation detection systems. This can, however, result in large and complex scenarios which are time consuming to model. A variety of approaches to radiation transport modeling exist with complementary strengths and weaknesses for different problems. This variety of approaches, and the development of promising new tools (such as ORNL’s ADVANTG) which combine benefits of multiple approaches, illustrates the need for a means of evaluating or comparing different techniques for radiation detection problems. This report presents a set of 9 benchmark problems for comparing different types of radiation transport calculations, identifying appropriate tools for classes of problems, and testing and guiding the development of new methods. The benchmarks were drawn primarily from existing or previous calculations with a preference for scenarios which include experimental data, or otherwise have results with a high level of confidence, are non-sensitive, and represent problem sets of interest to NA-22. From a technical perspective, the benchmarks were chosen to span a range of difficulty and to include gamma transport, neutron transport, or both and represent different important physical processes and a range of sensitivity to angular or energy fidelity. Following benchmark identification, existing information about geometry, measurements, and previous calculations were assembled. Monte Carlo results (MCNP decks) were reviewed or created and re-run in order to attain accurate computational times and to verify agreement with experimental data, when present. Benchmark information was then conveyed to ORNL in order to guide testing and development of hybrid calculations. The results of those ADVANTG calculations were then sent to PNNL for compilation. This is a report describing the details of the selected Benchmarks and results from various transport codes.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

BACT Simulation User Guide (Version 7.0)

This report documents the structure and operation of a simulation model of the Benchmark Active Control Technology (BACT) Wind-Tunnel Model. The BACT system was designed, built, and tested at NASA Langley Research Center as part of the Benchmark Models Program and was developed to perform wind-tunnel experiments to obtain benchmark quality data to validate computational fluid dynamics and computational aeroelasticity codes, to verify the accuracy of current aeroservoelasticity design and analysis tools, and to provide an active controls testbed for evaluating new and innovative control algorithms for flutter suppression and gust load alleviation. The BACT system has been especially valuable as a control system testbed.

Waszak, Martin R.↗

QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding

Quantum computing calibration depends on interpreting experimental data, and calibration plots provide the most universal human-readable representation for this task, yet no systematic evaluation exists of how well vision-language models (VLMs) interpret them. We introduce QCalEval, the first VLM benchmark for quantum calibration plots: 243 samples across 87 scenario types from 22 experiment families, spanning superconducting qubits and neutral atoms, evaluated on six question types in both zero-shot and in-context learning settings. The best general-purpose zero-shot model reaches a mean score of 72.3, and many open-weight models degrade under multi-image in-context learning, whereas frontier closed models improve substantially. A supervised fine-tuning ablation at the 9-billion-parameter scale shows that SFT improves zero-shot performance but cannot close the multimodal in-context learning gap. As a reference case study, we release NVIDIA Ising Calibration 1, an open-weight model based on Qwen3.5-35B-A3B that reaches 74.7 zero-shot average score.

Cao, Shuxiang↗

On the Fidelity of Computational Models for the Flow of Milled Loblolly Pine: A Benchmark Study on Continuum-Mechanics Models and Discrete-Particle Models

The upstream of bioenergy industry has suffered from unreliable operations of granular biomass feedstocks in handling equipment. Computational modeling, including continuum-mechanics models and discrete-particle models, offers insightful understandings and predictive capabilities on the flow of milled biomass and can assist equipment design and optimization. This paper presents a benchmark study on the fidelity of the continuum and discrete modeling approaches for predicting granular biomass flow. We first introduce the constitutive law of the continuum-mechanics model and the contact law of the coarse-grained discrete-particle model, with model parameters calibrated against laboratory characterization tests of the milled loblolly pine. Three classical granular material flow systems (i.e., a lab-scale rotating drum, a pilot-scale hopper, and a full-scale inclined plane) are then simulated using the two models with the same initial and boundary conditions as the physical experiments. The close agreement of the numerical predictions with the experimental measurements on the hopper mass flow rate, the hopper critical outlet width, the material stopping thickness on the inclined plane, and the dynamic angle of repose, clearly indicates that the two methods can capture the critical flow behavior of granular biomass. The qualitative comparison shows that the continuum-mechanics model outperforms in parameterization of materials and wall friction, and large-scale systems, while the discrete-particle model is more preferred for discontinuous flow systems at smaller scales. Industry stakeholders can use these findings as guidance for choosing appropriate numerical tools to model biomass material flow in part of the optimization of material handling equipment in biorefineries.

Jin, Wencheng↗

Two MCNP Models for Computational Performance Benchmarking

The purpose of this report is to describe two non-trivial MCNP models and execution configurations that are suitable to characterizing computational performance. To that end, this report uses a constructive solid geometry (CSG) representation of the Oak Ridge National Laboratory (ORNL) Pool Critical Assembly (PCA) [1–3] to perform a k-eigenvalue calculation and an unstructured mesh (UM) representation of the International Commission on Radiological Protection (ICRP) publication 145 (ICRP145) male human phantom [4, 5] to perform a fixed-source calculation assuming that a 1 MeV photon source is distributed throughout the phantom’s liver.

97 MATHEMATICS AND COMPUTING↗

Downfolding from ab initio to interacting model Hamiltonians: comprehensive analysis and benchmarking of the DFT+cRPA approach

Abstract Model Hamiltonians are regularly derived from first principles to describe correlated matter. However, the standard methods for this contain a number of largely unexplored approximations. For a strongly correlated impurity model system, here we carefully compare a standard downfolding technique with the best possible ground-truth estimates for charge-neutral excited-state energies and wave functions using state-of-the-art first-principles many-body wave function approaches. To this end, we use the vanadocene molecule and analyze all downfolding aspects, including the Hamiltonian form, target basis, double-counting correction, and Coulomb interaction screening models. We find that the choice of target-space basis functions emerges as a key factor for the quality of the downfolded results, while orbital-dependent double-counting corrections diminish the quality. Background screening of the Coulomb interaction matrix elements primarily affects crystal-field excitations. Our benchmark uncovers the relative importance of each downfolding step and offers insights into the potential accuracy of minimal downfolded model Hamiltonians.

Chemistry↗

A PROPOSED BENCHMARK PROBLEM FOR SCATTER CALCULATIONS IN RADIOGRAPHIC MODELLING

Code Validation is a permanent concern in computer modelling, and has been addressed repeatedly in eddy current and ultrasonic modeling. A good benchmark problem is sufficiently simple to be taken into account by various codes without strong requirements on geometry representation capabilities, focuses on few or even a single aspect of the problem at hand to facilitate interpretation and to avoid that compound errors compensate themselves, yields a quantitative result and is experimentally accessible. In this paper we attempt to address code validation for one aspect of radiographic modeling, the scattered radiation prediction. Many NDT applications can not neglect scattered radiation, and the scatter calculation thus is important to faithfully simulate the inspection situation. Our benchmark problem covers the wall thickness range of 10 to 50 mm for single wall inspections, with energies ranging from 100 to 500 keV in the first stage, and up to 1 MeV with wall thicknesses up to 70 mm in the extended stage. A simple plate geometry is sufficient for this purpose, and the scatter data is compared on a photon level, without a film model, which allows for comparisons with reference codes like MCNP. We compare results of three Monte Carlo codes (McRay, Sindbad and Moderato) as well as an analytical first order scattering code (VXI), and confront them to results obtained with MCNP. The comparison with an analytical scatter model provides insights into the application domain where this kind of approach can successfully replace Monte-Carlo calculations.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

An improved semi-global intrinsic kinetics model for high temperature carbon oxidation

Measurements of the oxidation rates of various forms of carbon (soot, graphite, coal char) have often shown an unexplained attenuation with increasing temperatures in the vicinity of 2000 K, even when accounting for diffusional transport limitations and gas-phase chemical effects (e.g. CO 2 dissociation). With the development of oxy-fuel combustion approaches for pulverized coal utilization with carbon capture, high particle temperatures are readily achieved in sufficiently oxygen-enriched environments. Here, in this work, a new semi-global intrinsic kinetics model for high temperature carbon oxidation is created by starting with a previously developed 5-step mechanism that was shown to reproduce all major known trends in carbon oxidation, except for its high temperature kinetic falloff, and incorporating a recently discovered surface oxide decomposition step. The predictions of this new model are benchmarked by deploying the kinetic model in a steady-state reacting particle code (SKIPPY) and comparing the simulated results against a carefully measured set of pulverized coal char combustion temperature measurements over a wide range of oxygen concentrations in N 2 and CO 2 environments. The results show that the inclusion of the spontaneous surface oxide decomposition reaction step significantly improves predictions at high particle temperatures. Furthermore, the simulations reveal that O atoms released from the oxide decomposition step enhance the radical pool in the near-surface region and within the particle interior itself. Incorporation of literature rates for O and OH reactions with the carbon surface results in a reduction in the predicted radical pool concentrations and a very minor enhancement of the overall carbon oxidation rate.

33 ADVANCED PROPULSION SYSTEMS↗

Final Report on DTRA Basic Research Project #BRCALL08-Per3-C-2-0006 "High-Z Non-Equilibrium Physics and Bright X-ray Sources with New Laser Targets"

This project had two major goals. Final Goal: obtain spectrally resolved, absolutely calibrated x-ray emission data from uniquely uniform mm-scale near-critical-density high-Z plasmas not in local thermodynamic equilibrium (LTE) to benchmark modern detailed atomic physics models. Scientific significance: advance understanding of non-LTE atomic physics. Intermediate Goal: develop new nano-fabrication techniques to make suitable laser targets that form the required highly uniform non-LTE plasmas when illuminated by high-intensity laser light. Scientific significance: advance understanding of nano-science. The new knowledge will allow us to make x-ray sources that are bright at the photon energies of most interest for testing radiation hardening technologies, the spectral energy range where current x-ray sources are weak. All project goals were met.

74 ATOMIC AND MOLECULAR PHYSICS↗

Challenging Geostatistical Methods To Represent Heterogeneity in CO2 Reservoirs Under Residual Trapping

Abstract Geostatistical methods based on two-point spatial-bivariate statistics have been used to model heterogeneity within computational studies of the dispersion of contaminants in groundwater reservoirs and the trapping of CO2 in geosequestration reservoirs. The ability of these methods to represent fluvial architecture, commonly occurring in such reservoirs, has been questioned. We challenged a widely used two-point spatial-bivariate statistical method to represent fluvial heterogeneity in the context of representing how reservoir heterogeneity affects residual trapping of CO2 injected for geosequestration. A more rigorous model for fluvial architecture was used as the benchmark in these studies. Both the geostatistically generated model and the benchmark model were interrogated, and metrics for the connectivity of high-permeability preferential flow pathways were quantified. Computational simulations of CO2 injection were performed, and metrics for CO2 dynamics and trapping were quantified. All metrics were similar between the two models. The percentage of high-permeability cells in spanning connected clusters (percolating clusters) was similar because percolation is strongly dependent upon proportions, and the same proportion of higher permeability cross-strata was specified in generating both models. The CO2 plume dynamics and residual trapping metrics were similar because they are largely controlled by the occurrence of percolating clusters. The benchmark model represented more features of the fluvial architecture and, depending on context, representing those features may be quite important, but the simpler geostatistical model was able to adequately represent fluvial reservoir architecture within the context and within the scope of the parameters represented here.

defects, mechanical behavior, carbon sequestration↗

Lifetime measurements of excited states in neutron-rich 53 Ti: Benchmarking effective shell-model interactions

Level lifetimes of the yrast (5/2 - ) to 13/2 - states in the neutron-rich nucleus 53 Ti, produced in a multinucleon-transfer reaction, have been measured for the first time. The recoil distance Doppler-shift method was employed and lifetimes of the excited states were extracted by a lineshape analysis aided by GEANT4-based Monte-Carlo simulations. The experiment was performed at the Grand Accelerateur National d'Ions Lourds facility in Caen, France, by using the Advanced Gamma Tracking Array for the gamma-ray detection coupled to the large-acceptance variable mode spectrometer for an event-by-event particle identification and the Cologne plunger for deep-inelastic reactions. Reduced transition probabilities, deduced from the lifetimes, give new information on the nuclear structure of 53 Ti, and are used to benchmark different shell-model calculations using established interactions in the fp shell.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Generation IV Benchmarking of TRISO Fuel Performance Models Under Accident Conditions Final Report

The Generation IV International Forum (GIF) is a co-operative international endeavor of fourteen members organized to carry out the research and development needed to establish the feasibility and performance capabilities of the next generation nuclear energy systems. GIF selected six reactor technologies, amongst which is the Very High Temperature Reactor (VHTR) that is primarily dedicated to the cogeneration of electricity and hydrogen. The technical basis for VHTR is the tristructural isotropic (TRISO)-coated particle fuel, the graphite as the core structure, helium coolant, as well as the dedicated core layout and lower power density to removal decay heat in a natural way. At the heart of safety features of the VHTR concept lie the TRISO fuel particles that are designed to keep their structural integrity and retain fission products at temperatures up to 1600°C. As part as the design and future operation of VHTRs, a key aspect is the accurate prediction of fuel performance under irradiation and accident conditions. Modeling and simulation allow prediction of TRISO fuel behavior when subject to neutron flux and in high temperature accident scenarios. The refinement of the fuel performance models and codes is performed by comparison to in-pile and out-of-pile experimental data that reproduce the expected irradiation conditions in high temperature gas-cooled reactors (HTGRs). Historically, the International Atomic Energy Agency (IAEA) developed a benchmark dedicated to the validation of predictive methods for fuel and fission product behavior through the Coordinated Research Program CRP-2 (IAEA, 1997). CRP-2 was later updated to cover fuel fabrication, quality assurance, irradiation performance, safety testing, and spent fuel. The scope of the resulting CRP-6 benchmarks focused on HTGR fuel performance and fission product release (IAEA, 2012). Taking advantage of additional TRISO fuel fabrication, irradiation, and safety testing campaigns, GIF launched a Generation IV Benchmarking of TRISO Fuel Performance Models under Accident Conditions in late 2015. This GIF benchmark is a three-year program steered by Idaho National Laboratory (INL, USA). The other participants include the Japan Atomic Energy Agency (JAEA, Japan) and the Korea Atomic Energy Research Institute (KAERI, Korea). The objectives of the benchmark are to: follow on the IAEA CRP benchmarks, (2) model fission product release under accident conditions, (3) compare results obtained by the fuel performance modeling codes of the benchmark participants, and (4) compare these code predictions to experimental data. Safety tests chosen for modeling include the first and second experiments of the Advanced Gas Reactor program (AGR-1 and AGR-2) and the High Flux Reactor (HFR) EU1bis experiment. This report presents the results obtained by the three research institutions using their respective fuel performance modeling codes. Comparisons of the corresponding fission product release predictions are made with experimental data from AGR-1, AGR-2, and HFR-EU1bis. The benchmark results show good agreements between all participants but also show a general trend of over-prediction of the experimental release data, which is mainly attributed to the use of over-estimated diffusion coefficients.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Results of the GABLS3 diurnal-cycle benchmark for wind energy applications

We present results of the GABLS3 model intercomparison benchmark revisited for wind energy applications. The case consists of a diurnal cycle, measured at the 200-m tall Cabauw tower in the Netherlands, including a nocturnal low-level jet. The benchmark includes a sensitivity analysis of WRF simulations using two input meteorological databases and five planetary boundary-layer schemes. A reference set of mesoscale tendencies is used to drive microscale simulations using RANS k-ϵ and LES turbulence models. The validation is based on rotor-based quantities of interest. Cycle-integrated mean absolute errors are used to quantify model performance. The results of the benchmark are used to discuss input uncertainties from mesoscale modelling, different meso-micro coupling strategies (online vs offline) and consistency between RANS and LES codes when dealing with boundary-layer mean flow quantities. Altogether, all the microscale simulations produce a consistent coupling with mesoscale forcings.

17 WIND ENERGY↗

Benchmarking a Defeatured Geant4 NIF Model with HTOAD and HNED Neutron Data

This study benchmarks a simplified Geant4 NIF Target Chamber (TC) model against foil measurements in two instruments: the HTOAD and HNED Snout. The number of product atoms per source neutron per gram ( N 0 /n/g ) is compared across multiple locations and material configurations. The model reproduces high energy threshold reactions dominated by 14 MeV neutrons to within a few percent, depending on location. Discrepancies occur where scattering and moderation are significant for low energy threshold reactions. The defeatured Geant4 model achieves comparable statistical precision with ∼400 times less CPU time than a full fidelity TC model in MCNP. The results can be produced locally on a laptop without the need for high performance computing. Low energy biasing can be improved by preserving room return pathways and implementing IRDFF cross section evaluations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Using multiple high-resolution datasets to benchmark the energy exascale earth system model (E3SM) for renewable resource assessment

The United States is accelerating its shift toward a renewable energy system. However, renewable resources, which harness energy from the Earth system, are susceptible to both present-day climate variability and future climate change. For example, variations in regional climate can alter renewable energy production patterns and site viability. The use of high-resolution climate model projections can therefore facilitate and may be critical to long-term planning of renewable energy investments. However, climate models must first be validated for renewable resource assessment. This research employs multiple high-spatiotemporal-resolution datasets to assess the capability of the Department of Energy’s (DOE) Energy Exascale Earth System Model version 2 North American Regionally Refined Model (E3SMv2-NARRM) for predicting multi-year climatological values of solar and wind energy capacity factors in the continental U.S., with a focus on regional and seasonal variability. Present-day E3SMv2-NARRM simulations are compared with reported utility-scale production data obtained from the Energy Information Administration (EIA). In addition, E3SMv2-NARRM data are evaluated against non-climate benchmark models from the National Renewable Energy Laboratory, including the Wind Integration National Dataset Toolkit and the National Solar Radiation Database (NSRDB), as well as three wind energy datasets from PLUSWIND. Our analysis indicates that solar capacity factors from E3SM closely match those from the NSRDB dataset. However, both datasets tend to overestimate values by 10% in comparison to EIA data. Furthermore, biases in wind capacity factors within E3SM are notably pronounced in the West Coast regions, where the seasonal cycle diverges from EIA data.

Energy forecasting, Capacity factor, Renewable ene↗