Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Feasibility of quasi-random band model in evaluating atmospheric radiance

The use of the quasi-random band model in evaluating upwelling atmospheric radiation is investigated. The spectral transmittance and total band adsorptance are evaluated for selected molecular bands by using the line by line model, quasi-random band model, exponential sum fit method, and empirical correlations, and these are compared with the available experimental results. The atmospheric transmittance and upwelling radiance were calculated by using the line by line and quasi random band models and were compared with the results of an existing program called LOWTRAN. The results obtained by the exponential sum fit and empirical relations were not in good agreement with experimental results and their use cannot be justified for atmospheric studies. The line by line model was found to be the best model for atmospheric applications, but it is not practical because of high computational costs. The results of the quasi random band model compare well with the line by line and experimental results. The use of the quasi random band model is recommended for evaluation of the atmospheric radiation.

Tiwari, S. N.↗

Status, Challenges, and Plans for Consistent Modeling and Evaluation of Fission Data: Nubar, PFNS, and FPY [Slides]

This presentation discusses the importance of consistent evaluations for fission observables. Currently, prompt fission quantities (e.g., neutron multiplicity, neutron energy spectrum, and fission product yields) are evaluated independently of one another, either with separate models or data-only/non-model evaluations (few – if any – shared model inputs) and inconsistencies can arise in evaluated data (e.g. NSE 190, 258 (2018)). They have found that consistent, model-based evaluations lead to more physical constraints on free parameters in the model, more robust predictions for other observables and other isotopes, that there are correlations between the uncertainties on different observables, and that uncertainties can be propagated to unmeasured quantities. They are working on developing models to the point where fission observables are calculated at the quality needed for an evaluation. They are striving to consistently calculate fission observables, such as $\bar{v}$, PFNS, and FPY, to a quality suitable for an evaluation. Current evaluations for fission observables use separate models for each observable (with minimal shared input) or are purely data driven. LANL models, such as the Monte Carlo CGMF and deterministic BeoH, have different strengths; the similar fission fragment initial conditions and decay models leads to consistency between the two codes (FPY can be connected to $\bar{v}$ and PFNS, even when calculated by different codes). Work is underway by Lovell to optimize the models and Neudecker is working to perform detailed uncertainty quantification. The PFNS, in particular, presents a significant challenge.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Technical Note: AQMEII4 Activity 1: Evaluation of Wet and Dry Deposition Schemes as an Integral Part of Regional-scale Air Quality Models

We present in this technical note the research protocol for phase 4 of the Air Quality Model Evaluation International Initiative (AQMEII4). This research initiative is divided into two activities, collectively having three goals: (i) to define the current state of the science with respect to representations of wet and especially dry deposition in regional models, (ii) to quantify the extent to which different dry deposition parameterizations influence retrospective air pollutant concentration and flux predictions, and (iii) to identify, through the use of a common set of detailed diagnostics, sensitivity simulations, model evaluation, and reduction of input uncertainty, the specific causes for the current range of these predictions. Activity 1 is dedicated to the diagnostic evaluation of wet and dry deposition processes in regional air quality models (described in this paper), and Activity 2 to the evaluation of dry deposition point models against ozone flux measurements at multiple towers with multiyear observations (to be described in future submissions as part of the special issue on AQMEII4). The scope of this paper is to present the scientific protocols for Activity 1, as well as to summarize the technical information associated with the different dry deposition approaches used by the participating research groups of AQMEII4. In addition to describing all common aspects and data used for this multi-model evaluation activity, most importantly, we present the strategy devised to allow a common process-level comparison of dry deposition obtained from models using sometimes very different dry deposition schemes. The strategy is based on adding detailed diagnostics to the algorithms used in the dry deposition modules of existing regional air quality models, in particular archiving diagnostics specific to land use–land cover (LULC) and creating standardized LULC categories to facilitate cross-comparison of LULC-specific dry deposition parameters and processes, as well as archiving effective conductance and effective flux as means for comparing the relative influence of different pathways towards the net or total dry deposition. This new approach, along with an analysis of precipitation and wet deposition fields, will provide an unprecedented process-oriented comparison of deposition in regional air quality models. Examples of how specific dry deposition schemes used in participating models have been reduced to the common set of comparable diagnostics defined for AQMEII4 are also presented.

Air Quality Model Evaluation International Initiat↗

Can 3-D models explain the observed fractions of fossil and non-fossil carbon in and near Mexico City?

Abstract. A 3-D chemistry-transport model has been applied to the Mexico City metropolitan area to investigate the origin of elevated levels of non-fossil (NF) carbonaceous aerosols observed in this highly urbanized region. High time resolution measurements of the fine aerosol concentration and composition, and 12 or 24 h integrated 14C measurements of aerosol modern carbon have been performed in and near Mexico City during the March 2006 MILAGRO field experiment. The non-fossil carbon fraction (fNF), which is lower than the measured modern fraction (fM) due to the elevated 14C in the atmosphere caused by nuclear bomb testing, is estimated from the measured fM and the source-dependent information on modern carbon enrichment. The fNF contained in PM1 total carbon analyzed by a US team (f TC NF ) ranged from 0.37 to 0.67 at the downtown location, and from 0.50 to 0.86 at the suburban site. Substantially lower values (i.e. 0.24–0.49) were found for PM10 filters downtown by an independent set of measurements (Swiss team), which are inconsistent with the modeled and known differences between the size ranges, suggesting higher than expected uncertainties in the measurement techniques of 14C. An increase in the non-fossil organic carbon (OC) fraction (f OC NF ) by 0.10–0.15 was observed for both sets of filters during periods with enhanced wildfire activity in comparison to periods when fires were suppressed by rain, which is consistent with the wildfire impacts estimated with other methods. Model results show that the relatively high fraction of nonfossil carbon found in Mexico City seems to arise from the combination in about equal proportions of regional biogenic SOA, biomass burning POA and SOA, as well as non-fossil urban POA and SOA. Predicted spatial and temporal variations for f OC NF are similar to those in the measurements between the urban vs. suburban sites, and high-fire vs. low-fire periods. The absolute modeled values of f OC NF are consistent with the Swiss dataset but lower than the US dataset. Resolving the 14C measurement discrepancies is necessary for further progress in model evaluation. The model simulations that included secondary organic aerosol (SOA) formation from semi-volatile and intermediate volatility (S/IVOC) vapors showed improved closure for the total OA mass compared to simulations which only included SOA from VOCs, providing a more realistic basis to evaluate the fNF predictions. f OC NF urban sources of modern carbon are important in reducing or removing the difference in fNF between model and measurements, even though they are often neglected on the interpretation of 14C datasets. An underprediction of biomass burning POA by the model during some mornings also explains a part of the model-measurement differences. The fNF of urban POA and SOA precursors is an important parameter that needs to be better constrained by measurements. Performing faster ( 3 h) 14C measurements in future campaigns is critical to further progress in this area. To our knowledge this is the first time that radiocarbon measurements are used together with aerosol mass spectrometer (AMS) organic components to assess the performance of a regional model for organic aerosols.

09 BIOMASS FUELS↗

WE-E-18A-09: Application of a Channelized Hotelling Observer Model to Evaluate Angiographic Imaging Equipment

Purpose: To develop and apply an observer model to objectively evaluate and compare the performance of different angiographic imaging equipment and acquisition variables. Methods: Image Acquisition— Iodine-based phantoms were created with target diameters: 0.5, 1, 2, and 4 mm. The phantoms were imaged using both planes of a bi-plane angiography system with detector pixel dimensions 0.1542 mm{sup 2} and 0.1842 mm{sup 2}, respectively. All four phantoms were imaged with magnification factors 1.5, 1.25 and 1 and with the large and small focal spots. Phantom position and the dose per frame (0.12– 0.24 μ Gy/frame) were varied for a single phantom size, magnification and focal spot. Observer Model— For each experimental condition, 1200 signal-present and signal-absent images were acquired and a detectability index (d') was calculated with a Gabor-channelized Hotelling observer (CHO) model. Detectability indices were evaluated as a function of dose, phantom size, and magnification. The model was then applied to compare d' of the two imaging planes and focal spots. Uncertainty in d' was estimated by bootstrapping the data and by examining the shift-variance of systems. Results: Detectability indices varied linearly with magnification and the square root of dose. For the 2 and 4 mm phantoms, d' varied linearly with diameter. For the 0.5 and 1 mm phantoms, d' expectedly deviated from this linear relationship due to substantial detector and focal spot blurring of the phantoms. The small focal spot yielded up to 50% greater d' values than the large focal spot. For the two detectors, differences in d' did not exceed the estimated ∼7% error. Conclusions: The detectability indices predictably scaled with dose, diameter, magnification, and focal spot size and serve to validate the model. Results demonstrate statistically similar target detectability for both investigated detectors, despite differences in pixel dimensions. This CHO model provides a framework to evaluate angiographic system performance.

60 APPLIED LIFE SCIENCES↗

Managing complexity in simulations of land surface and near-surface processes

Increasing computing power and the growing role of simulation in Earth systems science have led to an increase in the number and complexity of processes in modern simulators. We present a multiphysics framework that specifies interfaces for coupled processes and automates weak and strong coupling strategies to manage this complexity. Process management is enabled by viewing the system of equations as a tree, where individual equations are associated with leaf nodes and coupling strategies with internal nodes. A dynamically generated dependency graph connects a variable to its dependencies, streamlining and automating model evaluation, easing model development, and ensuring models are modular and flexible. Additionally, the dependency graph is used to ensure that data requirements are consistent between all processes in a given simulation. Here we discuss the design and implementation of these concepts within the Arcos framework, and demonstrate their use for verification testing and hypothesis evaluation in numerical experiments.

54 ENVIRONMENTAL SCIENCES↗

Technical note: AQMEII4 Activity 1: evaluation of wet and dry deposition schemes as an integral part of regional-scale air quality models

We present in this technical note the research protocol for phase 4 of the Air Quality Model Evaluation International Initiative (AQMEII4). This research initiative is divided into two activities, collectively having three goals: (i) to define the current state of the science with respect to representations of wet and especially dry deposition in regional models, (ii) to quantify the extent to which different dry deposition parameterizations influence retrospective air pollutant concentration and flux predictions, and (iii) to identify, through the use of a common set of detailed diagnostics, sensitivity simulations, model evaluation, and reduction of input uncertainty, the specific causes for the current range of these predictions. Activity 1 is dedicated to the diagnostic evaluation of wet and dry deposition processes in regional air quality models (described in this paper), and Activity 2 to the evaluation of dry deposition point models against ozone flux measurements at multiple towers with multiyear observations (to be described in future submissions as part of the special issue on AQMEII4). The scope of this paper is to present the scientific protocols for Activity 1, as well as to summarize the technical information associated with the different dry deposition approaches used by the participating research groups of AQMEII4. In addition to describing all common aspects and data used for this multi-model evaluation activity, most importantly, we present the strategy devised to allow a common process-level comparison of dry deposition obtained from models using sometimes very different dry deposition schemes. The strategy is based on adding detailed diagnostics to the algorithms used in the dry deposition modules of existing regional air quality models, in particular archiving diagnostics specific to land use–land cover (LULC) and creating standardized LULC categories to facilitate cross-comparison of LULC-specific dry deposition parameters and processes, as well as archiving effective conductance and effective flux as means for comparing the relative influence of different pathways towards the net or total dry deposition. This new approach, along with an analysis of precipitation and wet deposition fields, will provide an unprecedented process-oriented comparison of deposition in regional air quality models. Examples of how specific dry deposition schemes used in participating models have been reduced to the common set of comparable diagnostics defined for AQMEII4 are also presented.

54 ENVIRONMENTAL SCIENCES↗

Review of Hydrologic Models for Evaluating Use of Remote Sensing Capabilities

Hydrologic models most commonly used by federal agencies for hydrologic forecasting are reviewed. Six catchment models and one snow accumulation and ablation model are reviewed. Information on the structure, parameters, states, and required inputs is presented in schematic diagrams and in tables. The primary and secondary roles of parameters and state variables with respect to their function in the models are identified. The information will be used to evaluate the usefulness of remote sensing capabilities in the operational use of hydrologic models.

Peck, E. L.↗

Multi-scale predictions of massive conifer mortality due to chronic temperature rise

Global temperature rise and extremes accompanying drought threaten forests and their associated climatic feedbacks. Our ability to accurately simulate drought-induced forest impacts remains highly uncertain in part owing to our failure to integrate physiological measurements, regional-scale models, and dynamic global vegetation models (DGVMs). Here we show consistent predictions of widespread mortality of needleleaf evergreen trees (NET) within Southwest USA by 2100 using state-of-the-art models evaluated against empirical data sets. Experimentally, dominant Southwest USA NET species died when they fell below predawn water potential (Ψ pd ) thresholds (April–August mean) beyond which photosynthesis, hydraulic and stomatal conductance, and carbohydrate availability approached zero. The evaluated regional models accurately predicted NET Ψ pd , and 91% of predictions (10 out of 11) exceeded mortality thresholds within the twenty-first century due to temperature rise. The independent DGVMs predicted ≥50% loss of Northern Hemisphere NET by 2100, consistent with the NET findings for Southwest USA. Notably, the global models underestimated future mortality within Southwest USA, highlighting that predictions of future mortality within global models may be underestimates. Taken together, the validated regional predictions and the global simulations predict widespread conifer loss in coming decades under projected global warming.

59 BASIC BIOLOGICAL SCIENCES↗

Localized dynamic subgrid closure for simulation of magnetohydrodynamic turbulence

A local dynamic kinetic energy model (LDKM) for large-eddy simulation (LES) of magnetohydrodynamic (MHD) turbulence is proposed. The proposed MHD turbulence model evaluates all model coefficients locally and dynamically without any ad hoc averaging. This model also does not assume low magnetic Reynolds numbers. The turbulent residual-helicity effect ({alpha}-effect) appearing in the magnetic induction equation is successfully modeled. For validation, high-Re decaying isotropic decay turbulence with and without a mean magnetic field are studied using LES. The effect of rotation is also studied. For the case without rotation, it is observed that the energy spectrum follows a k{sup -5/3} law. For the case with rotation, it is shown that two mechanisms, phase scrambling due to frame rotation and Joule dissipation, are competing, and two distinct regimes with respect to rotation rate are observed. There is a critical rotation rate at which the energy decays most in MHD turbulence. It is also shown that this MHD-LDKM model is applicable to wide variety of high/low magnetic Reynolds number applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Accelerating Scientific Simulations with Bi-Fidelity Weighted Transfer Learning

High-fidelity modeling is an essential design tool for many engineering applications. However, for complex systems, computational cost can be a limiting factor. Analyzing parameter sensitivity, uncertainty quantification, and design optimization require many model evaluations. Surrogate models are often used to develop the relationship between model parameters and quantities of interest. However, in the case of complex systems, surrogate models require several degrees of freedom and, thus, a large number of data points to determine the correct dependencies. For many applications, this may be prohibitively expensive. The reduction of computational requirements can be achieved by leveraging low-fidelity models. Low-fidelity models represent the system at a coarser resolution with the advantage of computational efficiency. Therefore, a bi-fidelity modeling paradigm, which augments the accuracy of a low-fidelity model in a computationally efficient manner by invoking limited runs of a high-fidelity model, can be leveraged to sufficiently balance the accuracy and computational requirements. In this work, a bi-fidelity weighted transfer learning method using neural networks was applied to a computational fluid dynamics heat transfer modeling problem. The transfer learning advantage was investigated as a function of hyperparameters. Our main finding is that the use of a bi-fidelity modeling paradigm achieves accuracy close to that of a high-fidelity Gaussian process model while significantly reducing computational cost. The bi-fidelity model achieves comparable performance with 90 high-fidelity samples-that is, 60% less than the samples needed to achieve similar accuracy without the use of bi-fidelity modeling,

Borowiec, Katarzyna↗

Comprehensive Neural Posterior Estimation for Galaxy-Galaxy Strong Lensing

We present a deep learning model based on neural posterior estimation (NPE) for comprehensive extraction of astrophysical parameters from galaxy-scale strong gravitational lenses. The unprecedentedly large amount of galaxy-scale strong lenses expected in future cosmological surveys (${\cal O}(10^5)$) promises to enable valuable statistical constraints in various studies ranging from galaxy formation to the nature of dark matter, but it also poses a significant challenge for traditional modelling pipelines. To this end, our automated model includes several new, state-of-the-art features and approaches leveraging the framework of simulation-based inference (SBI). We infer a total of 20 parameters describing the mass and light profiles of both lens and source galaxies, using simulated raw multi-band data modelled under noise and observing conditions expected by the Legacy Survey of Space and Time (LSST), with its summary statistics generated by a residual network. We examine the efficacy of multi-band data in extracting nearly 20 model parameters simultaneous from strong lensing images including lens light. Finally, We perform a comprehensive set of diagnostics for SBI models, evaluating the model's prediction accuracy, stability, and uncertainty quantification.

Zhao, Roy J. [Chicago U., KICP]↗

Flash Diffusivity Measurement of Semi-Porous Insulation Material Intended for Radioisotope Thermoelectric Generators

This research involves the analysis of flash-heating data pursuant to find the thermal diffusivity of a semi-porous insulation material (Min-K) that is being considered for use in a radioisotope thermoelectric generator (RTG). An RTG uses radioactive nuclear fuel to produce electricity through a temperature difference imposed on a bimetallic thermocouple. Insulation is required to protect sensitive equipment from high temperatures and to conserve heat in the fuel. Using flash diffusivity temperature data, various simulations were fitted in order to find the most appropriate heat transfer model for the experiments. Four models were allowed to compete and the standard deviation of the residuals were compared for each model in evaluating model performance. The residuals are simply the difference in temperatures between the measurements and the mathematical models at each measurement point.

McMasters, Robert L.↗

Arctic Mixed-Phase Cloud Base Ice Precipitation Properties During the M-PACE Field Campaign

Cloud-climate feedbacks are still the greatest source of uncertainty in current climate projections. Arctic clouds, which are predominantly stratiform and supercooled, often long-lived, and nearly-continuously precipitate ice particles, contribute roughly 10% of the uncertainty attributed to the global cloud feedback. This Arctic cloud uncertainty is driven by incomplete observational and theoretical knowledge required to estimate and explain the state and active processes occurring in those clouds. A focus on ice precipitation properties at Arctic cloud base rather than the surface deconfounds the product of cloud condensate sink processes from the influence of the atmospheric thermodynamic state below cloud base, rendering cloud-base properties a more appealing target for inference and evaluation of model simulations. This dataset provides a set of 25 samples from the M-PACE field campaign, all of which were retrieved using the synthesis of ARM radar and lidar measurements. The retrieved ice precipitation variables in this dataset include, among others, the ice number concentration, water content, PSD parameters, precipitation rate, mass-weighted fall velocity, vertical air motion, and effective radius, all of which are highly valuable for model evaluation and a general understanding of polar cloud sink processes. Each variable sample includes its mean value and associated uncertainty. Additional variables based on ARM measurements (liquid layer statistics, etc.) are included in this dataset as well. The retrieval algorithm and analysis of this dataset are described in Silber (JGR, 2023, https://doi.org/10.1029/2022JD038202).

54 ENVIRONMENTAL SCIENCES↗

Arctic Mixed-Phase Cloud Base Ice Precipitation Properties Over the NSA Site

Cloud-climate feedbacks are still the greatest source of uncertainty in current climate projections. Arctic clouds, which are predominantly stratiform and supercooled, often long-lived, and nearly continuously precipitate ice particles, contribute roughly 10% of the uncertainty attributed to the global cloud feedback. This arctic cloud uncertainty is driven by incomplete observational and theoretical knowledge required to estimate and explain the state and active processes occurring in those clouds. A focus on ice precipitation properties at arctic cloud base rather than the surface deconfounds the product of cloud condensate sink processes from the influence of the atmospheric thermodynamic state below cloud base, rendering cloud-base properties a more appealing target for inference and evaluation of model simulations. This data set provides more than 1800 samples of cloud-base ice precipitation properties over Utqiagvik, North Slope of Alaska, all of which were retrieved using the synthesis of ARM radar and lidar measurements. The retrieved ice precipitation variables in this data set include, among others, the ice number concentration, water content, PSD parameters, precipitation rate, mass-weighted fall velocity, vertical air motion, and effective radius, all of which are highly valuable for model evaluation and a general understanding of polar cloud sink processes. Each variable sample includes its mean value and associated uncertainty. Additional variables based on ARM measurements (liquid layer statistics, etc.) are included in this data set. The retrieval algorithm and analysis of this data set are described in Silber (JGR, 2023, https://doi.org/10.1029/2022JD038202).

54 ENVIRONMENTAL SCIENCES↗

An Empirical Jet-Surface Interaction Noise Model with Temperature and Nozzle Aspect Ratio Effects

An empirical model for jet-surface interaction (JSI) noise produced by a round jet near a flat plate is described and the resulting model evaluated. The model covers unheated and hot jet conditions (1 less than or equal to jet total temperature ratio less than or equal to 2.7) in the subsonic range (0.5 less than or equal to M(sub a) less than or equal to 0.9), surface lengths 0.6 less than or equal to (axial distance from jet exit to surface trailing edge (inches)/nozzle exit diameter) less than or equal to 10, and surface standoff distances (0 less than or equal to (radial distance from jet lipline to surface (inches)/axial distance from jet exit to surface trailing edge (inches)) less than or equal to 1) using only second-order polynomials to provide predictable behavior. The JSI noise model is combined with an existing jet mixing noise model to produce exhaust noise predictions. Fit quality metrics and comparisons to between the predicted and experimental data indicate that the model is suitable for many system level studies. A first-order correction to the JSI source model that accounts for the effect of nozzle aspect ratio is also explored. This correction is based on changes to the potential core length and frequency scaling associated with rectangular nozzles up to 8:1 aspect ratio. However, more work is needed to refine these findings into a formal model.

Noise↗