Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58

Is the assumption of normality or log-normality for continuous response data critical for benchmark dose estimation?

Continuous responses (e.g. body weight) are widely used in risk assessment for determining the benchmark dose (BMD) which is used to derive a U.S. EPA reference dose. One critical question that is not often addressed in dose–response assessments is whether to model the continuous data as normally or log-normally distributed. Additionally, if lognormality is assumed, and only summarized response data (i.e., mean ± standard deviation) are available as is usual in the peer-reviewed literature, the BMD can only be approximated. In this study, using the “hybrid” method and relative deviation approach, we first evaluate six representative continuous dose–response datasets reporting individual animal responses to investigate the impact on BMD/BMDL estimates of (1) the distribution assumption and (2) the use of summarized versus individual animal data when a log-normal distribution is assumed. We also conduct simulation studies evaluating model fits to various known distributions to investigate whether the distribution assumption has influence on BMD/BMDL estimates. Our results indicate that BMDs estimated using the hybrid method are more sensitive to the distribution assumption than counterpart BMDs estimated using the relative deviation approach. The choice of distribution assumption has limited impact on the BMD/BMDL estimates when the within dose-group variance is small, while the lognormality assumption is a better choice for relative deviation method when data are more skewed because of its appropriateness in describing the relationship between mean and standard deviation. Additionally, the results suggest that the use of summarized data versus individual response data to characterize log-normal distributions has minimal impact on BMD estimates. - Highlights: • We investigate to what extent the distribution assumption can affect BMD estimates. • Both real data analysis and simulation study are conducted. • BMDs estimated using hybrid method are more sensitive to distribution assumption. • Summarized continuous data are adequate for BMD estimation.

60 APPLIED LIFE SCIENCES↗

Are Atmospheric Models Too Cold in the Mountains? The State of Science and Insights from the SAIL Field Campaign

Mountains play an outsized role in water resource availability, and the amount and timing of water they provide depend strongly on temperature. To that end, we ask the question: How well are atmospheric models capturing mountain temperatures? We synthesize results showing that high-resolution, regionally relevant climate models produce 2-m air temperature (T2m) measurements colder than what is observed (a “cold bias”), particularly in snow-covered midlatitude mountain ranges during winter. We find common cold biases in 44 studies across global mountain ranges, including single-model and multimodel ensembles. We explore the factors driving these biases and examine the physical mechanisms, data limitations, and observational uncertainties behind T2m. Our analysis suggests that the biases are genuine and not due to observation sparsity or resolution mismatches. Cold biases occur primarily on mountain peaks and ridges, whereas valleys are often warm biased. Our literature review suggests that increasing model resolution does not clearly mitigate the bias. By analyzing data from the Surface Atmosphere Integrated Field Laboratory (SAIL) field campaign in the Colorado Rocky Mountains, we test various hypotheses related to cold biases and find that local wind circulations, longwave (LW) radiation, and surface-layer parameterizations contribute to the T2m biases in this particular location. We conclude by emphasizing the value of coordinated model evaluation and development efforts in heavily instrumented mountain locations for addressing the root cause(s) of T2m biases and improving predictive understanding of mountain climates.

54 ENVIRONMENTAL SCIENCES↗

Coupled SAM/Griffin Model of a Reference Fluoride-Salt-Cooled High-Temperature Reactor for Multi-Physics Simulations

A multi-physics coupled simulation model of a reference pebble bed fluoride-salt-cooled high-temperature reactor (PB-FHR) has been developed with SAM and Griffin computer codes for transient safety analysis. The reference problem of a prototypical reactor design serves as the foundation for the U.S. NRC (Nuclear Regulatory Commission) to verify the adequacy of computer codes and evaluation models for specific reactor types. In this work, the previously developed SAM model for PB-FHR primary loop has been updated for the coupled simulation. The updated SAM PB-FHR model includes a 2-D axial symmetric core region and external core components in 0-D/1-D. In addition to the primary loop, a detailed model of the RCCS (reactor cavity cooling system) is added. The 2-D and 1-D domains are tightly coupled using the single-solve approach developed in SAM. In the pebble bed region, the SAM multiscale explicit pebble model is applied to calculate the pebble and TRISO fuel kernel temperatures. The Griffin model used in this work is based on a model developed at Idaho National Laboratory in collaboration with the U.S. NRC. The Griffin neutronics model and SAM thermal hydraulics model is coupled with the Comprehensive Reactor Analysis Bundle (CRAB or alternately BlueCRAB) application. Both steady-state and transient scenarios are simulated to demonstrate the model's suitability for multi-physics simulations of PB-FHR transients.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

On the use of radiative surface temperature to estimate sensible heat flux over sparse shrubs in Nevada

The accurate partitioning of available energy into sensible and latent heat flux is crucial to the understanding of surface atmosphere interactions. This issue is more complicated in arid and semi arid regions where the relative contribution to surface fluxes from the soil and vegetation may vary significantly throughout the day and throughout the season. A three component model to estimate sensible heat flux over heterogeneous surfaces is presented. The surface was represented with two adjacent compartments. The first compartment is made up of two components, shrubs and shaded soil, the second of open 'illuminated' soil. Data collected at two different sites in Nevada (U.S.) during the Summers of 1991 and 1992 were used to evaluate model performance. The results show that the present model is sufficiently general to yield satisfactory results for both sites.

Chehbouni, A.↗

Adoption of image-driven machine learning for microstructure characterization and materials design: A Perspective

Microstructure characterization enables the development of structure-processing-property relationships critical to several research areas within the broad field of materials science, from alloy design to the assessment of corrosion resistance, and failure analysis. Conventional approaches to material characterization have relied on either qualitative inference by the human ex-pert or software applications that can extract high-level features from images, such as boundary segmentation, average grain diameter, etc. Such approaches rely heavily on subject matter expert user intervention and knowledge of what phases or more generally, what microstructural features, are of interest. The recent surge in the adoption of machine learning techniques to address problems in materials engineering has brought with it an increased interest and application of Image Driven Machine Learning (IDML) approaches. In this work, we review the applications of IDML to the field of materials characterization. A canonical hierarchy of stages is defined, which when put sequentially together completes an IDML study: problem definition, dataset building, model selection and training, model evaluation, and integration with existing instrumentation or simulation workflow. The studies reviewed in this work are analyzed from the perspective of each of these stages. Such a review permits agranular assessment of the field, for example the impact of IDML on materials characterization at the nanoscale, the size of a typical dataset required to train a semantic segmentation model on electron microscopy images, ubiquitousness of transfer learning in the domain, etc. Finally, we discuss the importance of interpretability and explainability in the field of IDML for materials characterization, and provide an overview of two emerging techniques in the field: semantic segmentation and generative adversarial networks.

Baskaran, Arun↗

An expert system for a distributed real-time trainer

The problem addressed by this expert system concerns the expansion of capability of a Real Time Trainer for the Spacelab flight crew. As requirements for more models or fidelity are placed upon the system, expansion is necessary. The simulator can be expanded using a larger processor or by going to a distributed system and expand by adding additional processors. The distributed system is preferable because it is more economical and can be expanded in a more incremental manner. An expert system was developed to evaluate modeling and timing capability within a real time training simulator. The expert system is based upon a distributed configuration. Components of the modeled system are control tasks, network tasks, emulator tasks, processors, displays, and a network. The distributed module expert system (DMES) allows the configuring of processors, tasks, display use, keyboard use, and selection of alternate methods to update the data buffer. Modules can be defined with execution occurring in a specific processor on a network. The system consists of a knowledge front end editor to interactively generate or update the knowledge base, an inference engine, a display module, and a recording module.

Purinton, Steven C.↗

Dynamic modeling of orographically induced precipitation

Local orography governs the triggering of cloud formation and the enhancement of processes such as condensation and hydrometeor nucleation and growth in mountainous regions. Intense, lengthy precipitation events are typical upwind of the topographic divide, with sharply decreasing magnitude and duration on the lee side. Differences in mean annual precipitation of several hundred percent between windward slopes of orographic barriers and adjacent valleys or lee side slopes are not unusual. Because much of the streamflow in areas such as the western United States is derived from mountainous areas that are remote and often poorly instrumented, modeling of orographic precipitation has important implications for water resources management. Models of orographically induced precipitation differ by their treatment of atmospheric dynamics and by the extent to which they rely on bulk parameterization of cloud and precipitation physics. Adiabatic ascent and a direct proportionality between efficiency and orographically magnified updrafts are the most frequent assumptions in orographic precipitation modeling. Space-time discretization (i.e., resolution) is a major issue because of the high spatial variability of orographic precipitation. For a specific storm, relative errors as large as 50 to 100% are common in the forecast/hindcast of precipitation intensity and can be even larger in the case of catastrophic storms. When monthly or seasonal timescales are used to evaluate model performance, the magnitude of such errors decreases dramatically, reaching values as low as 10 to 15%. Current research is focusing on the development of data assimilation techniques to incorporate radar and satellite observations, and on the development of aggregation and disaggregation methodologies to address the implications of modeling a multiscale problem at restricted spatial and temporal resolutions.

Barros, Ana Paula↗

Global Sensitivity Analysis Using the Ultra‐Low Resolution Energy Exascale Earth System Model

Abstract For decades, Arctic temperatures have increased twice as fast as average global temperatures. As a first step toward quantifying parametric uncertainty in Arctic climate, we performed a variance‐based global sensitivity analysis (GSA) using a fully coupled, ultra‐low resolution (ULR) configuration of version 1 of the U.S. Department of Energy's Energy Exascale Earth System Model (E3SMv1). Specifically, we quantified the sensitivity of six quantities of interests (QOIs), which characterize changes in Arctic climate over a 75 year period, to uncertainties in nine model parameters spanning the sea ice, atmosphere, and ocean components of E3SMv1. Sensitivity indices for each QOI were computed with a Gaussian process emulator using 139 random realizations of the random parameters and fixed preindustrial forcing. Uncertainties in the atmospheric parameters in the Cloud Layers Unified by Binormals (CLUBB) scheme were found to have the most impact on sea ice status and the larger Arctic climate. Our results demonstrate the importance of conducting sensitivity analyses with fully coupled climate models. The ULR configuration makes such studies computationally feasible today due to its low computational cost. When advances in computational power and modeling algorithms enable the tractable use of higher‐resolution models, our results will provide a baseline that can quantify the impact of model resolution on the accuracy of sensitivity indices. Moreover, the confidence intervals provided by our study, which we used to quantify the impact of the number of model evaluations on the accuracy of sensitivity estimates, have the potential to inform the computational resources needed for future sensitivity studies.

54 ENVIRONMENTAL SCIENCES↗

Coupled SAM/Griffin Model of a Reference Pebble Bed High-Temperature Gas Cooled Reactor for Multi-Physics Simulations

An effort has been dedicated to developing a reference model for multi-physics coupled simulation of the pebble bed high-temperature gas-cooled reactor (PB-HTGR) with SAM and Griffin computer codes for safety analysis and licensing purpose. The reference problem of a prototypical reactor design serves as the foundation for the U.S. NRC (Nuclear Regulatory Commission) to verify the adequacy of computer codes and evaluation models for specific reactor types. In this work, a SAM model of the HTR-PM reactor has been developed based on publicly available design information and the multi-dimensional Pronghorn model developed by Idaho National Laboratory. The SAM HTR-PM model includes a multi-dimensional core region and 0- D/1-D fluid components. Additionally, a simplified air RCCS loop is modeled for decay heat removal. The Griffin model is based on a recent work by Idaho National Laboratory. The coupling between the models is achieved through the MOOSE MultiApp system. Both steady-state and transient scenarios were simulated to demonstrate the coupled model’s capability for multi-physics simulations.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Vertical structure and physical processes of the Madden-Julian oscillation: Exploring key model physics in climate simulations

Aimed at reducing deficiencies in representing the Madden-Julian oscillation (MJO) in general circulation models (GCMs), a global model evaluation project on vertical structure and physical processes of the MJO was coordinated. In this paper, results from the climate simulation component of this project are reported. Here, it is shown that the MJO remains a great challenge in these latest generation GCMs. The systematic eastward propagation of the MJO is only well simulated in about one fourth of the total participating models. The observed vertical westward tilt with altitude of the MJO is well simulated in good MJO models but not in the poor ones. Damped Kelvin wave responses to the east of convection in the lower troposphere could be responsible for the missing MJO preconditioning process in these poor MJO models. Several process-oriented diagnostics were conducted to discriminate key processes for realistic MJO simulations. While large-scale rainfall partition and low-level mean zonal winds over the Indo-Pacific in a model are not found to be closely associated with its MJO skill, two metrics, including the low-level relative humidity difference between high- and low-rain events and seasonal mean gross moist stability, exhibit statistically significant correlations with the MJO performance. It is further indicated that increased cloud-radiative feedback tends to be associated with reduced amplitude of intraseasonal variability, which is incompatible with the radiative instability theory previously proposed for the MJO. Finally, results in this study confirm that inclusion of air-sea interaction can lead to significant improvement in simulating the MJO.

54 ENVIRONMENTAL SCIENCES↗

Baseflow Identification via Explainable AI With Kolmogorov‐Arnold Networks

Abstract Hydrological models often involve constitutive laws that may not be optimal in every application. We propose to replace such laws with the Kolmogorov‐Arnold networks (KANs), a class of neural networks designed to identify symbolic expressions. We demonstrate KAN's potential on the problem of baseflow identification, a notoriously challenging task plagued by significant uncertainty. KAN‐derived functional dependencies of the baseflow components on the aridity index outperform their original counterparts; they demonstrate that water availability, rather than potential evapotranspiration, drives baseflow by constraining actual evapotranspiration under arid conditions. On a test set, they increase the Nash‐Sutcliffe efficiency (NSE) by 65%, decrease the root mean squared error by 29%, and increase the Kling‐Gupta efficiency by 34%. This superior performance is achieved while reducing the number of fitting parameters from three to two. Next, we use data from 378 catchments across the continental United States to refine the water‐balance equation at the mean‐annual scale. The KAN‐derived equations based on the refined water balance outperform both the current aridity index model, with up to a 105% increase in NSE, and the KAN‐derived equations based on the original water balance. While the performance of our model and tree‐based machine learning methods is similar, KANs offer the advantage of simplicity and transparency and require no specific software or computational tools. This case study focuses on the aridity index formulation, but the approach is flexible and transferable to other hydrological processes. Plain Language Summary Equations used in hydrologic model are often suboptimal, resulting in reduced prediction accuracy and efficiency. We implemented Kolmogorov‐Arnold networks (KAN), a machine learning algorithm for deriving symbolic formulations, to estimate groundwater recharge and showed that it outperforms an existing state‐of‐the‐art semi‐empirical formulation. In hydrology, Nash‐Sutcliffe efficiency (NSE), root mean squared error (RMSE), and Kling‐Gupta efficiency (KGE) are commonly used to evaluate model performance. Higher NSE and KGE values indicate better performance, while lower RMSE values are preferable. Our results show that NSE increased by 71%, RMSE decreased by 32%, and KGE improved by 25%. In addition, KAN identifies an optimal functional form and can be used to derive new analytical formulas using the prior knowledge. The KAN‐inspired equation outperformed the original formulation and reduced the fitting parameters. Furthermore, we refined the water‐balance equation at the mean‐annual scale and showed that, based on the new water‐balance equation, KAN can derive new formulations that are superior to the original aridity index formulations (up to 105% increase in NSE) and KAN‐derived equations based on the original water balance. These findings highlight the significant potential of KAN to advance the scientific understanding of a wide range of hydrologic processes. Key Points Kolmogorov‐Arnold networks (KANs) enhance interpretability of machine‐learned hydrological models KAN‐derived symbolic formulations outperform state‐of‐the‐art semi‐empirical aridity indices KAN‐identified functional form yields an analytical index with fewer fitting parameters and improved performance

baseflow↗

Building America Case Study: Assessment of a Hybrid Retrofit Gas Water Heater

This project completed a modeling evaluation of a hybrid gas water heater that combines a reduced capacity tankless unit with a downsized storage tank. This product would meet a significant market need by providing a higher efficiency gas water heater solution for retrofit applications while maintaining compatibility with the half-inch gas lines and standard B vents found in most homes. The TRNSYS simulation tool was used to model a base case 0.60 EF atmospheric gas storage water, a 0.82 EF non-condensing gas tankless water heater, an existing (high capacity) hybrid unit on the market, and an alternative hybrid unit with lower storage volume and reduced gas input requirements.

ARBI↗

Using regime analysis to identify the contribution of clouds to surface temperature errors in weather and climate models

Many global circulation models (GCMs) exhibit a persistent bias in the 2 m temperature over the midlatitude continents, present in short-range forecasts as well as long-term climate simulations. A number of hypotheses have been proposed, revolving around deficiencies in the soil–vegetation–atmosphere energy exchange, poorly resolved low-level boundary-layer clouds or misrepresentations of deep-convective storms. A common approach to evaluating model biases focuses on the model-mean state. However, this makes difficult an unambiguous interpretation of the origins of a bias, given that biases are the result of the superposition of impacts of clouds and land-surface deficiencies over multiple time steps. This article presents a new methodology to objectively detect the role of clouds in the creation of a surface warm bias. A unique feature of this study is its focus on temperature-error growth at the time-step level. It is shown that compositing the temperature-error growth by the coinciding bias in total downwelling radiation provides unambiguous evidence for the role that clouds play in the creation of the surface warm bias during certain portions of the day. Furthermore, the application of an objective cloud-regime classification allows for the detection of the specific cloud regimes that matter most for the creation of the bias. We applied this method to two state-of-the-art GCMs that exhibit a distinct warm bias over the Southern Great Plains of the USA. Our analysis highlights that, in one GCM, biases in deep-convective and low-level clouds contribute most to the temperature-error growth in the afternoon and evening respectively. In the second GCM, deep clouds persist too long in the evening, leading to a growth of the temperature bias. In conclusion, the reduction of the temperature bias in both models in the morning and the growth of the bias in the second GCM in the afternoon could not be assigned to a cloud issue, but are more likely caused by a land-surface deficiency.

54 ENVIRONMENTAL SCIENCES↗

Neural Active Manifolds: Nonlinear Dimensionality Reduction for Uncertainty Quantification

We present a new approach for nonlinear dimensionality reduction, specifically designed for computationally expensive mathematical models. We leverage autoencoders to discover a one-dimensional neural active manifold (NeurAM) capturing the model output variability, through the aid of a simultaneously learnt surrogate model with inputs on this manifold. Our method only relies on model evaluations and does not require the knowledge of gradients. The proposed dimensionality reduction framework can then be applied to assist outer loop many-query tasks in scientific computing, like sensitivity analysis and multifidelity uncertainty propagation. In particular, we prove, both theoretically under idealized conditions, and numerically in challenging test cases, how NeurAM can be used to obtain multifidelity sampling estimators with reduced variance by sampling the models on the discovered low-dimensional and shared manifold among models. Several numerical examples illustrate the main features of the proposed dimensionality reduction strategy and highlight its advantages with respect to existing approaches in the literature.

Autoencoders↗

Propulsion-airframe interactions predictability

An illustration of the possible magnitude of the transonic performance prediction problem is shown. These results derived from a careful and systematic set of tests run on a jet effects model, an aerodynamic and force model, and a pressure model, evaluated test techniques and wind tunnel effects. The results were then carefully compared with flight test data where significant discrepancies in zero-lift drag were observed. A vigorous review of the test techniques identified a number of potential error sources. Some of these were thought to be aircraft roughness and protuberances, tunnel anomalies, sting/model support corrections, and hot gas effects. In addition, there were questions on model metric splitline location and magnitude of corrections for scale and Reynolds number. Flight test inlet/engine characteristics were thought to differ, also, from those of the calibration settings in the ground facilities.

Smith, R. H.↗

Design and dispatch optimization of packaged ice storage systems within a connected community

The traditional implementation of cool thermal energy storage (CTES) must be reimagined within the context of a dynamic grid and smart buildings operating as connected communities. As most buildings do not operate central chillers or connect to district cooling loops, this necessitates a broader use of packaged CTES. Our objective is to begin answering the question of how such packaged CTES should be implemented within a connected community. We do so by presenting a simulation–optimization workflow employing building energy modeling software and a mixed-integer linear program to design and dispatch a packaged CTES technology to achieve minimum total annual cost. We demonstrate this methodology on a seven-building case study using current utility rates and find that total annual cooling energy costs can be reduced by 17.8% compared to baseline, after accounting for the cost of storage. We perform three parametric sensitivity studies to evaluate modeling assumptions and obtain the prioritization of storage procurement as a function of annualized life-cycle cost of storage. We find that a community optimization approach provides significantly different results than individual building optimizations and provides greater savings compared to baseline.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Robust Importance Sampling for Bayesian Model Calibration with Spatio-Temporal Data

This paper addresses two challenges in Bayesian calibration: 1) computational speed of existing sampling algorithms, and 2) calibration with spatio-temporal responses. The commonly used Markov Chain Monte Carlo (MCMC) approaches require many sequential model evaluations making the computational expense prohibitive. This paper proposes an efficient sampling algorithm: iterative importance sampling with genetic algorithm (IISGA). While iterative importance sampling enables computational efficiency, the genetic algorithm enables robustness by preventing sample degeneration and avoids getting stuck in multimodal search spaces. An inflated likelihood further enables robustness in high-dimensional parameter spaces by enlarging the target distribution. Spatio-temporal data complicate both surrogate modeling, which is necessary for expensive computational models, and the likelihood estimation. In this work, singular value decomposition is investigated for reducing the high-dimensional field data to a lower-dimensional space prior to Bayesian calibration. Then the likelihood is formulated and Bayesian inference is performed in the lower-dimension, latent space. An illustrative example is provided to demonstrate IISGA relative to existing sampling methods, and then IISGA is employed to calibrate a thermal battery model with 26 uncertain calibration parameters and spatio-temporal response data.

97 MATHEMATICS AND COMPUTING↗