Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76

Evaluation of an open forecasting challenge to assess skill of West Nile virus neuroinvasive disease prediction

Abstract Background West Nile virus (WNV) is the leading cause of mosquito-borne illness in the continental USA. WNV occurrence has high spatiotemporal variation, and current approaches to targeted control of the virus are limited, making forecasting a public health priority. However, little research has been done to compare strengths and weaknesses of WNV disease forecasting approaches on the national scale. We used forecasts submitted to the 2020 WNV Forecasting Challenge, an open challenge organized by the Centers for Disease Control and Prevention, to assess the status of WNV neuroinvasive disease (WNND) prediction and identify avenues for improvement. Methods We performed a multi-model comparative assessment of probabilistic forecasts submitted by 15 teams for annual WNND cases in US counties for 2020 and assessed forecast accuracy, calibration, and discriminatory power. In the evaluation, we included forecasts produced by comparison models of varying complexity as benchmarks of forecast performance. We also used regression analysis to identify modeling approaches and contextual factors that were associated with forecast skill. Results Simple models based on historical WNND cases generally scored better than more complex models and combined higher discriminatory power with better calibration of uncertainty. Forecast skill improved across updated forecast submissions submitted during the 2020 season. Among models using additional data, inclusion of climate or human demographic data was associated with higher skill, while inclusion of mosquito or land use data was associated with lower skill. We also identified population size, extreme minimum winter temperature, and interannual variation in WNND cases as county-level characteristics associated with variation in forecast skill. Conclusions Historical WNND cases were strong predictors of future cases with minimal increase in skill achieved by models that included other factors. Although opportunities might exist to specifically improve predictions for areas with large populations and low or high winter temperatures, areas with high case-count variability are intrinsically more difficult to predict. Also, the prediction of outbreaks, which are outliers relative to typical case numbers, remains difficult. Further improvements to prediction could be obtained with improved calibration of forecast uncertainty and access to real-time data streams (e.g. current weather and preliminary human cases). Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗

STUDIES OF VARIABILITY IN PROTO-PLANETARY NEBULAE. II. LIGHT AND VELOCITY CURVE ANALYSES OF IRAS 22272+5435 AND 22223+4327

We have carried out a detailed observational study of the light, color, and velocity variations of two bright, carbon-rich proto-planetary nebulae, IRAS 22223+4327 and 22272+5435. The light curves are based upon our observations from 1994 to 2011, together with published data by Arkhipova and collaborators. They each display four significant periods, with primary periods for IRAS 22223+4327 and 22272+5435 being 90 and 132 days, respectively. For each of them, the ratio of secondary to primary period is 0.95, a value much different from that found in Cepheids, but which may be characteristic of post-asymptotic giant branch (AGB) stars. Fewer significant periods are found in the smaller radial velocity data sets, but they agree with those of the light curves. The color curves generally mimic the light curves, with the objects reddest when faintest. A comparison in seasons when there exist contemporaneous light, color, and velocity curves reveals that the light and color curves are in phase, while the radial velocity curves are {approx}0.25 P out of phase with the light curves. Thus they differ from what is seen in Cepheids, in which the radial velocity curve is 0.50 P out of phase with the light curve. Comparison of the observed periods and amplitudes with those of post-AGB pulsation models shows poor agreement, especially for the periods, which are much longer than predicted. These observational data, particularly the contemporaneous light, color, and velocity curves, provide an excellent benchmark for new pulsation models of cool stars in the post-AGB, proto-planetary nebula phase.

79 ASTRONOMY AND ASTROPHYSICS↗

A Metallic Fuel Performance Benchmark Problem Based on the IFR-1 Experiment

Metallic nuclear fuels are an active area of research and development for use in advanced reactors. Robust and accurate metallic fuel performance models are necessary for the design, safety analysis, and licensing of such reactors. However, metallic fuel performance models are not as mature as ceramic fuel performance models and require additional development. In order to better gauge the accuracy of existing models, identify high-priority models for development, and quantify the improvements made by future model development, ORNL has developed a benchmark problem based on the IFR-1 experiment. This work gathers all relevant information on the IFR-1 experiment and uses it to develop the benchmark problem. It simulates the problem using the fuel performance code BISON and compare the results to post-irradiation examination data from the experiment. Then a sensitivity analysis is performed on the BISON model to determine the benchmark problem’s sensitivity to uncertainty in the input parameters. The results suggest that BISON’s mechanical models require additional development. Plastic deformation is under-predicted in the cladding and axial swelling is over-predicted in the fuel. These problems may be related. Additionally, there is a bias in the temperature which may be a result of uncertainty in the input parameters rather than an issue with the fuel performance models.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Improving E3SM Land Model Photosynthesis Parameterization via Satellite SIF, Machine Learning, and Surrogate Modeling

The parameterization of key photosynthesis parameters is one of the key uncertain sources in modeling ecosystem gross primary productivity (GPP). Solar-induced chlorophyll fluorescence (SIF) offers a good proxy for GPP since it marks the actual process of photosynthesis; while machine learning (ML) provides a robust approach to model the GPP-SIF relationship. Here, we trained the boosted regressing tree (BRT) and the Random Forest ML models with Greenhouse Gases Observing Satellite SIF data and in situ GPP observations from 49 eddy covariance towers. These trained ML GPP-SIF models were fed into the Energy Exascale Earth System Model (E3SM) Land Model (ELM) to generate ELM-simulated global SIF estimates, which were then benchmarked against satellite SIF observations with a surrogate modeling approach. Our results indicated good modeling performance of the ML-based GPP-SIF relationship. The ELM model when fed with the ML GPP-SIF models also can well predict the spatial-temporal variations in SIF. We also found high model accuracy for the surrogate modeling. Model parameter sensitivity analysis suggested that the fraction of leaf nitrogen in RuBisCO (flnr) is the most sensitive parameter to the SIF; other sensitive parameters include the Ball-Berry stomatal conductance slope (mbbopt) and the vcmax entropy (vcmaxse). The posterior uncertainty in simulated GPP was greatly reduced after benchmarking, and the model produced improved spatial patterns of mean GPP relative to FLUXCOM GPP. Our integrated approach provides a new avenue for improving land models and using remote-sensing SIF, which can be further improved in the future with more ground- and satellite-based observations.

54 ENVIRONMENTAL SCIENCES↗

Deterministic Modeling of the High Temperature Test Reactor with DRAGON-HEXPEDITE

The Idaho National Laboratory (INL) is tasked with the development of reactor physics analysis capability of the Next Generation Nuclear Power (NGNP) project. In order to examine the INL’s current prismatic reactor analysis tools, the project is conducting a benchmark exercise based on modeling the High Temperature Test Reactor (HTTR). This exercise entails the development of a model for the initial criticality, a 19 fuel column thin annular core, and the fully loaded core critical condition with 30 fuel columns. Special emphasis is devoted to physical phenomena and artifacts in HTTR that are similar to phenomena and artifacts in the NGNP base design. The DRAGON code is used in this study since it offers significant ease and versatility in modeling prismatic designs. DRAGON can generate transport solutions via Collision Probability (CP), Method of Characteristics (MOC) and Discrete Ordinates (Sn). A fine group cross-section library based on the SHEM 281 energy structure is used in the DRAGON calculations. The results from this study show reasonable agreement in the calculation of the core multiplication factor with the MC methods, but a consistent bias of 2-3% with the experimental values is obtained. This systematic error has also been observed in other HTTR benchmark efforts and is well documented in the literature. The ENDF/B VII graphite and U 235 cross sections appear to be the main source of the error. The isothermal temperature coefficients calculated with the fully loaded core configuration agree well with other benchmark participants but are 40% higher than the experimental values. This discrepancy with the measurement partially stems from the fact that during the experiments the control rods were adjusted to maintain criticality, whereas in the model, the rod positions were fixed. In addition, this work includes a brief study of a cross section generation approach that seeks to decouple the domain in order to account for neighbor effects. This spectral interpenetration is a dominant effect in annular HTR physics. This analysis methodology should be further explored in order to reduce the error that is systematically propagated in the traditional generation of cross sections.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Aerothermal modeling program, phase 2

The main objectives of the Aerothermal Modeling Program, Phase 2 are: to develop an improved numerical scheme for incorporation in a 3-D combustor flow model; to conduct a benchmark quality experiment to study the interaction of a primary jet with a confined swirling crossflow and to assess current and advanced turbulence and scalar transport models; and to conduct experimental evaluation of the air swirler interaction with fuel injectors, assessments of current two-phase models, and verification the improved spray evaporation/dispersion models.

Mongia, H. C.↗

Development of a Benchmark Eddy Flux Evapotranspiration Dataset for Evaluation of Satellite-Driven Evapotranspiration Models Over the CONUS

A large sample of ground-based evapotranspiration (ET) measurements made in the United States, primarily from eddy covariance systems, were post-processed to produce a benchmark ET dataset. The dataset was produced primarily to support the intercomparison and evaluation of the OpenET satellite-based remote sensing ET (RSET) models and could also be used to evaluate ET data from other models and approaches. OpenET is a web-based service that makes field-delineated and pixel-level ET estimates from well-established RSET models readily available to water managers, agricultural producers, and the public. The benchmark dataset is composed of flux and meteorological data from a variety of providers covering native vegetation and agricultural settings. Flux footprint predictions were developed for each station and included static flux footprints developed based on average wind direction and speed, as well as dynamic hourly footprints that were generated with a physically based model of upwind source area. The two footprint prediction methods were rigorously compared to evaluate their relative spatial coverage. Data from all sources were post-processed in a consistent and reproducible manner including data handling, gap-filling, temporal aggregation, and energy balance closure correction. The resulting dataset included 243,048 daily and 5,284 monthly ET values from 194 stations, with all data falling between 1995 and 2021. We assessed average daily energy imbalance using 172 flux sites with a total of 193,021 days of data, finding that overall turbulent fluxes were understated by about 12% on average relative to available energy. Multiple linear regression analyses indicated that daily average latent energy flux may be typically understated slightly more than sensible heat flux. This dataset was developed to provide a consistent reference to support evaluation of RSET data being developed for a wide range of applications related to water accounting and water resources management at field to watershed scales.

54 ENVIRONMENTAL SCIENCES↗

Benchmarking impact hydrocodes in the strength regime: Implications for modeling deflection by a kinetic impactor

The Double Asteroid Redirection Test (DART) is a NASA-sponsored mission that will be the first direct test of the kinetic impactor technique for planetary defense. Here, the DART spacecraft will impact into Didymos-B, the moon of the binary system 65803 Didymos and the resulting period change will be measured from Earth. Impact simulations will be used to predict the crater size and momentum enhancement expected from the DART impact. Because the specific material properties (strength, porosity, internal structure) of the Didymos-B target are unknown, a wide variety of numerical simulations must be performed to better understand possible impact outcomes. This simulation campaign will involve a large parameter space being simulated using multiple different shock physics hydrocodes. In order to understand better the behaviors and properties of numerical simulation codes applicable to the DART impact, a benchmarking and validation program using different numerical codes to solve a set of standard problems was designed and implemented.

79 ASTRONOMY AND ASTROPHYSICS↗

Benchmark Progress towards TREAT M8CAL Core Loading to Support Validation of TREAT Operations

Benchmark experiment data is necessary to validate modeling and simulation activities to support Transient Reactor Experiment Test (TREAT) Facility restart and operations. Key measurements of interest include criticality, control rod worth, excess reactivity, and shutdown margin for varying core loading sizes. Benchmark evaluations are being developed according the guidelines provided in the International Handbook of Evaluated Reactor Physics Benchmark Experiments (IRPhEP Handbook) that can also support advanced modeling and simulation activities, new experiment design for advanced reactors and accident tolerant fuel, and potential low-enriched uranium conversion of the TREAT reactor. This summary discusses development activities supporting a benchmark evaluation of the TREAT M8CAL core loading, which will be submitted for peer-review and publication in the IRPhEP Handbook with the identifier TREAT-FUND-RESR-002. Detailed models of TREAT were developed and evaluated to prepare benchmark models of the minimum critical mass configuration. Calculations were performed using MCNP6.1 with ENDF/B-VII.1 nuclear data libraries. Discussion of results.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Benchmark Development for TREAT Minimum Critical Mass Core Loading to Support Validation of TREAT Operations

Benchmark experiment data is necessary to validate modeling and simulation activities to support Transient Reactor Experiment Test (TREAT) Facility restart and operations. Key measurements of interest include criticality, control rod worth, excess reactivity, and shutdown margin for varying core loading sizes. Benchmark evaluations are being developed according the guidelines provided in the International Handbook of Evaluated Reactor Physics Benchmark Experiments (IRPhEP Handbook) that can also support advanced modeling and simulation activities, new experiment design for advanced reactors and accident tolerant fuel, and potential low-enriched uranium conversion of the TREAT reactor. This summary discusses development activities supporting a benchmark evaluation of the TREAT minimum critical mass core loading, which will be submitted for peer-review and publication in the IRPhEP Handbook with the identifier TREAT-FUND-RESR-001. Detailed models of TREAT were developed and evaluated to prepare benchmark models of the minimum critical mass configuration. Calculations were performed using MCNP6.1 with ENDF/B-VII.1 nuclear data libraries. Discussion of results.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Characterizing the multisectoral impacts of future global hydrologic variability

There is significant uncertainty in how global water supply will evolve in the future, due to uncertain climate, socioeconomic, and land use change drivers and variability of hydrologic processes. It is critical to characterize the potential impacts of uncertainty in future water supply given its importance for food and energy production. In this work, we introduce a framework that integrates stochastic hydrology and human-environmental systems to characterize uncertainty in future water supply and its multisector impacts. We develop a global stochastic watershed model and demonstrate that this model can generate a large ensemble of realizations of basin-scale runoff with global coverage that preserves the mean, variance, and spatial correlation of a historical benchmark. We couple this model with a well-known human-environmental systems model to explore the impacts of runoff variability on the water and agricultural sectors across spatial scales. We find that the impacts of future hydrologic variability vary across sectors and regions. Impacts are felt most strongly in the water and agricultural sectors for basins that are expected to have unsustainable water use in the future, such as the Indus River basin. For this basin, we find that the variability in future irrigation water withdrawals and irrigated cropland increase over time due to uncertainty in renewable water supply. We also use the Indus basin to show how our stochastic ensemble can be leveraged to explore the global multisector consequences of local extreme runoff conditions. This work introduces a novel technique to explore the propagation of future hydrologic variability across human and natural systems and spatial scales.

54 ENVIRONMENTAL SCIENCES↗

Estimation of Participation Factors Using the Synchrosqueezed Wavelet Transform

This paper proposes a data-driven approach for estimating participation factors for a power system using only simulation results on selected disturbances. The approach is purely response-based and does not need a linearized system model for eigen-analysis, which makes it applicable to systems whose detailed, complete mathematical models are not available. Considering the unavoidable nonlinearity as exhibited in the transient period of a system response, the Synchrosqueezed Wavelet Transform is applied to simulated responses for modal analysis to obtain participation factors. Based on simulations of Kundur's two-area system using both the electromagnetic transient model and phasor model, the participation factors estimated by the proposed approach are compared with two other signal processing tools, the Prony analysis and continuous wavelet transform, and are also benchmarked with conventional model-based participation factors.

participation factor↗

DOES CRITICAL MASS DECREASE AS TEMPERATURE INCREASES: A REVIEW OF FIVE BENCHMARK EXPERIMENTS THAT SPAN A RANGE OF ELEVATED TEMPERATURES AND CRITICAL CONFIGURATIONS

Five sets of benchmark experiments are reviewed herein that cover a diverse set of fissile system configurations. The review specifically focused on the change in critical mass of these systems at elevated temperatures and the temperature reactivity coefficient ({alpha}{sub T}) on the system. Because plutonium-based critical benchmark experiments at varying temperatures were not found at the time this review was prepared, only uranium-based systems are included, as follows: (1) HEU-SOL-THERM-010 - UO{sub 2}F{sub 2} solutions with high U{sup 235} enrichment; (2) HEU-COMP-THERM-016 - uranium-graphite blocks with low U concentration; (3) LEU-COMP-THERM-032 - water moderated lattices of UO{sub 2} with stainless steel cladding, and intermediate U{sup 235} enrichment; (4) IEU-COMP-THERM-002 - water moderated lattices of annular UO{sub 2} with/without absorbers, and intermediate U{sup 235} enrichment; and (5) LEU-COMP-THERM-026 - water moderated lattices of UO{sub 2} at different pitches, and low U{sup 235} enrichment. In three of the five benchmarks (1, 3 and 5), modeling of the critical system at room temperature is conservative compared to modeling the system at elevated temperatures, i.e., a greater fissile mass is required at elevated temperature. In one benchmark (4), there was no difference in the fissile mass between the room temperature system and the system at the examined elevated temperature. In benchmark (2), the system clearly had a negative temperature reactivity coefficient. Some of the high temperature benchmark experiments were treated with appropriate (and comprehensive) adjustments to the cross section sets and thermal expansion coefficients, while other experiments were treated with partial adjustments. Regardless of the temperature treatment, modeling the systems at room temperature was found to be conservative for the examined systems, i.e., a smaller critical mass was obtained. While the five benchmarks presented herein demonstrate that, for the conditions examined, modeling of the systems at room temperature is conservative as compared to modeling the systems at elevated temperatures, it is possible to design a system in which the critical mass at room temperature is non-conservative compared to a system at elevated temperatures. As the temperature of the systems evaluated in this review was increased, the system's overall {alpha}{sub T} was negative at elevated temperatures. Furthermore, the review demonstrates that to accurate asses the effect of increased temperature on a system's k{sub eff}, changes in fissile, moderator, cladding, and, in some cases, structural material cross sections must be combined with other factors that influence reactivity, such as volumetric thermal expansion of fissile, moderating, reflector, and other interacting media. Altering the microscopic cross sections of fissile and moderating regions for temperature changes, without adjusting the corresponding densities at elevated temperatures can lead to an incorrect assessment of the impact of elevated temperature on a fissile system.

36 MATERIALS SCIENCE↗

Controlling Heteroepitaxy by Oxygen Chemical Potential: Exclusive Growth of (100) Oriented Ceria Nanostructures on Cu(111)

We present a novel and simple method for the preparation of a well-defined CeO 2 (100) model system on Cu(111) based on the adjustment of the Ce/O ratio during growth. The method yields micrometer-sized, several nanometers high, single-phase CeO 2 (100) islands with controllable size and surface termination that can be benchmarked against the known (111) nanostructured islands on Cu(111). We also demonstrate the ability to adjust the Ce to O stoichiometry from CeO 2 (100) (100% Ce 4+ ) to c-Ce 2 O 3 (100) (100% Ce 3+ ), which can be readily recognized by characteristic surface reconstructions observed by low-energy electron diffraction. Finally, the discovery of the highly stable CeO x (100) phase on a hexagonally close packed metal surface represents an unexpected growth mechanism of ceria on Cu(111), and it provides novel opportunities to prepare more elaborate models, benchmark surface chemical reactivity, and thus gain valuable insights into the redox chemistry of ceria in catalytic processes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

U.S. Solar Photovoltaic BESS System Cost Benchmark Q1 2020 Report

NREL has been modeling U.S. solar photovoltaic (PV) system costs since 2009. This year, our report benchmarks costs of U.S. PV for residential, commercial, and utility-scale systems, with and without storage, built in the first quarter of 2020 (Q1 2020). The methodology includes bottom-up accounting for all system and project-development costs incurred when installing residential, commercial, and utility-scale systems and it models the capital costs and levelized cost of energy for such systems.

14 SOLAR ENERGY↗

A hybrid calorimetry-simulation model of mixing enthalpy for molten salt

Calorimetric determination of enthalpies of mixing (ΔH mix ) in multicomponent molten salts is often interpreted using empirical models that lack physically meaningful parameters. However, for improving pyrochemical separation of spent nuclear fuel, where lanthanides are major fission products and critical elements, a deeper thermodynamic understanding of the link between excess thermodynamic properties and solvation structure is critically needed. In this work, we implement a hybrid and physics-informed framework, MIVM+Calorimetry+AIMD, which integrates experimentally measured ΔH mix (via high temperature drop calorimetry) with solvation structures from ab initio molecular dynamics (AIMD). This approach is demonstrated using LaCl 3 mixed with eutectic LiCl-KCl (58 mol% – 42 mol%) at 873 K and 1133 K. MIVM-derived parameters enable extrapolation of excess Gibbs energy and La 3+ activity across compositions. In contrast, direct ΔH mix predictions from AIMD and polarizable ion model simulations deviate significantly. By incorporating experimentally benchmarked solvation structures into an interpretable thermodynamic model, the MIVM+Calorimetry+AIMD formalism achieves higher accuracy and generalizable method for studying molten salts, offering a robust path for understanding and optimizing molten salt chemistry relevant to nuclear fuel cycles and separation science.

Goncharov, Vitaliy G. [Washington State Univ., Pul↗

Thermal Profiling of Residential Energy Use

This work describes a methodology for informing targeted demand-response (DR) and marketing programs that focus on the temperature-sensitive part of residential electricity demand. Our methodology uses data that is becoming readily available at utility companies-hourly energy consumption readings collected from "smart" electricity meters, as well as hourly temperature readings. To decompose individual consumption into a thermal-sensitive part and a base load (non-thermally-sensitive), we propose a model of temperature response that is based on thermal regimes, i.e., unobserved decisions of consumers to use their heating or cooling appliances. We use this model to extract useful benchmarks that compose thermal profiles of individual users, i.e., terse characterizations of the statistics of these users' temperature-sensitive consumption. We present example profiles generated using our model on real consumers, and show its performance on a large sample of residential users. This knowledge may, in turn, inform the DR program by allowing scarce operational and marketing budgets to be spent on the right users-those whose influencing will yield highest energy reductions-at the right time. We show that such segmentation and targeting of users may offer savings exceeding 100% of a random strategy.

Albert, A↗

Liquid Water: Obtaining the right answer for the right reasons

Water is ubiquitous on our planet and plays an essential role in many chemical and biological processes. Accurate models for water are crucial in understanding, controlling and predicting the physical and chemical properties of complex aqueous systems. Over the last few years we have been developing a molecular-level based approach for a macroscopic model for water that is based on the explicit description of the underlying intermolecular interactions between molecules in water clusters. In the absence of detailed experimental data for small water clusters, highly-accurate theoretical results are required to validate and parameterize model potentials. As an example of the benchmarks needed for the development of accurate models for the interaction between water molecules, for the most stable structure of (H2O)20 we ran a coupled-cluster calculation on the ORNL’s Jaguar petaflop computer that used over 100 TB of memory for a sustained performance of 487 TFLOP/s (double precision) on 96,000 processors, lasting for 2 hours. By this summer we will have studied multiple structures of both (H2O)20 and (H2O)30 and completed basis set and other convergence studies and anticipate the sustained performance rising close to 1 PFLOP/s.

aqueous↗