Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

Operational Dust Prediction

Over the last few years, numerical prediction of dust aerosol concentration has become prominent at several research and operational weather centres due to growing interest from diverse stakeholders, such as solar energy plant managers, health professionals, aviation and military authorities and policymakers. Dust prediction in numerical weather prediction-type models faces a number of challenges owing to the complexity of the system. At the centre of the problem is the vast range of scales required to fully account for all of the physical processes related to dust. Another limiting factor is the paucity of suitable dust observations available for model, evaluation and assimilation. This chapter discusses in detail numerical prediction of dust with examples from systems that are currently providing dust forecasts in near real-time or are part of international efforts to establish daily provision of dust forecasts based on multi-model ensembles. The various models are introduced and described along with an overview on the importance of dust prediction activities and a historical perspective. Assimilation and evaluation aspects in dust prediction are also discussed.

assimilation↗

North American Extreme Precipitation Events and Related Large-Scale Meteorological Patterns: a Review of Statistical Methods, Dynamics, Modeling, and Trends

This paper surveys the current state of knowledge regarding Large-Scale Meteorological Patterns (LSMPs) associated with short-duration (less than one week) extreme precipitation events over North America. In contrast to teleconnections, which are typically defined based on the characteristic spatial variations of a meteorological field or on the remote circulation response to a known forcing, LSMPs are defined relative to the occurrence of a specific phenomenon—here, extreme precipitation—and with an emphasis on the synoptic scales that have a primary influence in individual events, have medium-range weather predictability, and are well-resolved in both weather and climate models. For the LSMP relationship with extreme precipitation, we consider the previous literature with respect to definitions and data, dynamical mechanisms, model representation, and climate change trends. There is considerable uncertainty in identifying extremes based on existing observational precipitation data and some limitations in analyzing the associated LSMPs in reanalysis data. Many different definitions of “extreme” are in use, making it difficult to directly compare different studies. Dynamically, several types of meteorological systems—extratropical cyclones, tropical cyclones, mesoscale convective systems, and mesohighs—and several mechanisms—fronts, atmospheric rivers, and orographic ascent—have been shown to be important aspects of extreme precipitation LSMPs. The extreme precipitation is often realized through mesoscale processes organized, enhanced, or triggered by the LSMP. Understanding of model representation, trends, and projections for LSMPs is at an early stage, although some 4 promising analysis techniques have been identified and the LSMP perspective is useful for evaluating model dynamics.

Mathew Barlow↗

Evaluation of CMIP6 models in simulating the statistics of extreme precipitation over Eastern Africa

We report the Eastern Africa region experiences frequent extreme precipitation events that can cause destruction of property and environment, and loss of lives. Thus, there is a need to understand how these events may change in the future and how well the global climate models that are used to make projections can simulate precipitation extremes in this region before they can be used in downscaling or flood and drought impact assessment studies. In this work, we evaluated the ability of sixteen Coupled Model Intercomparison Project Phase 6 (CMIP6) models to simulate present-day precipitation extremes over the Eastern Africa region during the two rainy seasons (March–May and September–November). We used nine extreme precipitation indices (including seven (one) indices of wet (dry) extremes) defined by the Expert Team on Climate Change Detection and Indices. The CMIP6 models were evaluated against two gridded observation datasets: Global Precipitation Climatology Project One-Degree Daily Dataset and Tropical Rainfall Measuring Mission Multi-satellite Precipitation Analysis 3B42. Three model performance metrics (percentage bias, normalized root-mean-square error, and pattern correlation coefficient) were employed to further assess the strengths and weakness of the models. Our results show that the multi-model ensemble mean generally provides a better representation of observed precipitation and related extremes compared to individual models when considering all metrics and seasons. Several consistent biases are evident across CMIP6 models, which tend to overestimate the total-wet day precipitation and consecutive wet days, and underestimate very wet days and maximum 5-day precipitation in both seasons. Furthermore, no single model consistently performs best, model performance varies with the season and index under consideration and is generally independent of horizontal resolution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Prediction of cloud condensation nuclei activity for organic compounds using functional group contribution methods

A wealth of recent laboratory and field experiments demonstrate that organic aerosol composition evolves with time in the atmosphere, leading to changes in the influence of the organic fraction to cloud condensation nuclei (CCN) spectra. There is a need for tools that can realistically represent the evolution of CCN activity to better predict indirect effects of organic aerosol on clouds and climate. This work describes a model to predict the CCN activity of organic compounds from functional group composition. Following previous methods in the literature, we test the ability of semi-empirical group contribution methods in Köhler theory to predict the effective hygroscopicity parameter, kappa. However, in our approach we also account for liquid–liquid phase boundaries to simulate phase-limited activation behavior. Model evaluation against a selected database of published laboratory measurements demonstrates that kappa can be predicted within a factor of 2. Simulation of homologous series is used to identify the relative effectiveness of different functional groups in increasing the CCN activity of weakly functionalized organic compounds. Hydroxyl, carboxyl, aldehyde, hydroperoxide, carbonyl, and ether moieties promote CCN activity while methylene and nitrate moieties inhibit CCN activity. The model can be incorporated into scale-bridging test beds such as the Generator of Explicit Chemistry and Kinetics of Organics in the Atmosphere (GECKO-A) to evaluate the evolution of kappa for a complex mix of organic compounds and to develop suitable parameterizations of CCN evolution for larger-scale models.

54 ENVIRONMENTAL SCIENCES↗

Quantifying physical parameterization uncertainties associated with land-atmosphere interactions in the WRF model over Amazon

The Weather Research and Forecasting (WRF) model can be used to diagnose regional land-atmosphere (L-A) coupling strength in the absence of sufficient observations but subjected to uncertainties associated with model physical parameterizations. In this study, we propose a framework to quantify and reduce model physical parameterization uncertainties associated with surface fluxes and L-A coupling. An ensemble of WRF simulations with different physical schemes is used to simulate surface fluxes and land-atmosphere coupling strength over the Amazon region. The physical parameterizations investigated include cloud microphysics (MP), land surface processes (LSM), planetary boundary layer (PBL), surface layer (SL), and cumulus (CU). We perform 120 ensemble simulations using the WRF model and different combinations of six MPs, three LSMs, six PBLs and SLs and three CUs. The measurements from the GoAMAZON field campaign and satellite data are used to evaluate model performance. A Multi-way analysis of variance (ANOVA) approach is applied to quantify the relative importance of different physics processes on L-A coupling. The Tukey’s test is used to sort schemes that have no significant differences into one group. The suite of physics that result in the best simulations of the corresponding variables are selected based on the Taylor skill score. Results show that the relative importance of processes and their interaction vary with the variables of interest. For example, CU was the most important process in modulating soil moisture, 2 m-humidity, latent heat, and net radiation. LSM showed dominant effects on 2 m-temperature and also has the largest impact on sensible heat and the lifting condensation level. The best physical parameterization ensembles show much narrower ranges of the variables of interest than the priori ensemble. Results of this study show the roles of different physical processes in modulating L-A interactions, quantify model uncertainties from physical processes, and provide insights for improving the model physics parameterizations.

Wang, Chen↗

SPITZER TRANSITS OF THE SUPER-EARTH GJ1214b AND IMPLICATIONS FOR ITS ATMOSPHERE

We observed the transiting super-Earth exoplanet GJ1214b using warm Spitzer at 4.5 {mu}m wavelength during a 20 day quasi-continuous sequence in 2011 May. The goals of our long observation were to accurately define the infrared transit radius of this nearby super-Earth, to search for the secondary eclipse, and to search for other transiting planets in the habitable zone of GJ1214. We here report results from the transit monitoring of GJ1214b, including a reanalysis of previous transit observations by Desert et al. In total, we analyze 14 transits of GJ1214b at 4.5 {mu}m, 3 transits at 3.6 {mu}m, and 7 new ground-based transits in the I+z band. Our new Spitzer data by themselves eliminate cloudless solar composition atmospheres for GJ1214b, and methane-rich models from Howe and Burrows. Using our new Spitzer measurements to anchor the observed transit radii of GJ1214b at long wavelengths, and adding new measurements in I+z, we evaluate models from Benneke and Seager and Howe and Burrows using a {chi}{sup 2} analysis. We find that the best-fit model exhibits an increase in transit radius at short wavelengths due to Rayleigh scattering. Pure water atmospheres are also possible. However, a flat line (no atmosphere detected) remains among the best of the statistically acceptable models, and better than pure water atmospheres. We explore the effect of systematic differences among results from different observational groups, and we find that the Howe and Burrows tholin-haze model remains the best fit, even when systematic differences among observers are considered.

79 ASTRONOMY AND ASTROPHYSICS↗

Glass formulation and composition optimization with property models: A review

Abstract Glass is a versatile material with a remarkable history and many practical applications. It plays a critical role in our everyday lives, the advancement of science, and the development of many technologies. The Edisonian type trial‐and‐error method was commonly used for conventional design of glass compositions, which was time‐consuming and costly. With the urgent need to develop new glass compositions for technology applications rapidly, it has become necessary to develop precise property models with predictive powers using large databases and efficient formulation approaches. This paper reviews the design of glass compositions using these analytical and numerical models of composition–structure–property relations of glasses, some based on large databases and machine learning approaches. Aspects of data collection, model fitting, feature extraction, model evaluation, and uncertainty quantification will be covered. Furthermore, advances in the glass optimization framework and available tools are summarized with examples. The outlook and perspective for further glass property model development and formulation approaches are discussed.

Lu, Xiaonan↗

Covariance-Free Bifidelity Control Variates Importance Sampling for Rare Event Reliability Analysis

Multifidelity modeling has been steadily gaining attention as a tool to address the problem of exorbitant model evaluation costs that makes the estimation of failure probabilities a significant computational challenge for complex real-world problems, particularly when failure is a rare event. To implement multifidelity modeling, estimators that efficiently combine information from multiple models/sources are necessary. In past works, the variance reduction techniques of control variates (CV) and importance sampling (IS) have been leveraged for this task. In this paper, we present the CVIS framework—a creative take on a coupled CV and IS estimator for bifidelity reliability analysis. The framework addresses some of the practical challenges of the CV method by using an estimator for the control variate mean and sidestepping the need to estimate the covariance between the original estimator and the control variate through a clever choice for the tuning constant. Furthermore, the task of selecting an efficient IS distribution is also considered, with a view towards maximally leveraging the bifidelity structure and maintaining expressivity. Additionally, a diagnostic is provided that indicates both the efficiency of the algorithm as well as the relative predictive quality of the models utilized. Finally, the behavior and performance of the framework is explored through analytical and numerical examples.

Markov chain Monte Carlo↗

elm-diagnostics

elm-diagnostics is a Python package for computing diagnostic analyses and visualizations for the E3SM Land Model (ELM) component and is meant to support new feature development in ELM. The tool reads model history files and performs quantitative analyses including budget-closure checking, variable transformations, temporal aggregations, and statistical summaries to support model evaluation, validation, and scientific interpretation. The framework is designed for extensibility, with modular architecture enabling straightforward addition of new diagnostic methods, derived variables, analysis types, visualization approaches, and model-specific adaptations

Hoffman, Matt [Los Alamos National Laboratory]↗

The GGCMI Phase 2 Experiment: Global Gridded Crop Model Simulations Under Uniform Changes in CO2, Temperature, Water, and Nitrogen Levels (Protocol Version 1.0)

Concerns about food security under climate change motivate efforts to better understand future changes in crop yields. Process-based crop models, which represent plant physiological and soil processes, are necessary tools for this purpose since they allow representing future climate and management conditions not sampled in the historical record and new locations to which cultivation may shift. However, process-based crop models differ in many critical details, and their responses to different interacting factors remain only poorly understood. The Global Gridded Crop Model Intercomparison (GGCMI) Phase 2 experiment, an activity of the Agricultural Model Intercomparison and Improvement Project (AgMIP), is designed to provide a systematic parameter sweep focused on climate change factors and their interaction with overall soil fertility, to allow both evaluating model behavior and emulating model responses in impact assessment tools. In this paper we describe the GGCMI Phase 2 experimental protocol and its simulation data archive. A total of 12 crop models simulate five crops with systematic uniform perturbations of historical climate, varying CO2, temperature, water supply, and applied nitrogen (“CTWN”) for rainfed and irrigated agriculture, and a second set of simulations represents a type of adaptation by allowing the adjustment of growing season length. We present some crop yield results to illustrate general characteristics of the simulations and potential uses of the GGCMI Phase 2 archive. For example, in cases without adaptation, modeled yields show robust decreases to warmer temperatures in almost all regions, with a nonlinear dependence that means yields in warmer baseline locations have greater temperature sensitivity. Inter-model uncertainty is qualitatively similar across all the four input dimensions but is largest in high-latitude regions where crops may be grown in the future.

GGCMI↗

Collaborative Project: The problem of bias in defining uncertainty in computationally enabled strategies for data-driven climate model development. Final Technical Report.

The objective of the project is to develop strategies for better representing scientific sensibilities within statistical measures of model skill that then can be used within a Bayesian statistical framework for data-driven climate model development and improved measures of model scientific uncertainty. One of the thorny issues in model evaluation is quantifying the effect of biases on climate projections. While any bias is not desirable, only those biases that affect feedbacks affect scatter in climate projections. The effort at the University of Texas is to analyze previously calculated ensembles of CAM3.1 with perturbed parameters to discover how biases affect projections of global warming. The hypothesis is that compensating errors in the control model can be identified by their effect on a combination of processes and that developing metrics that are sensitive to dependencies among state variables would provide a way to select version of climate models that may reduce scatter in climate projections. Gabriel Huerta at the University of New Mexico is responsible for developing statistical methods for evaluating these field dependencies. The UT effort will incorporate these developments into MECS, which is a set of python scripts being developed at the University of Texas for managing the workflow associated with data-driven climate model development over HPC resources. This report reflects the main activities at the University of New Mexico where the PI (Huerta) and the Postdocs (Nosedal, Hattab and Karki) worked on the project.

58 GEOSCIENCES↗

Machine learning for the redox potential prediction of molecules in organic redox flow battery

Here, organic redox flow batteries (ORFB) are recognized as an innovative technology for the large-scale storage of renewable energy. The redox potential of organic redox-active molecules plays a vital role in their performance. Advanced screening techniques like high-throughput experiment and machine learning (ML) have significantly enhanced organic material performance and transformed the field of ORFB. However, the scarcity of experimental data poses a considerable challenge for ML model development in this domain. In our study, we developed lightweight graph-based Gaussian process regression (GPR) models with GPU-accelerated marginalized graph kernel and hybrid kernel to predict the redox potentials of organic redox-active molecules for ORFBs, specifically focusing on small datasets. To evaluate model accuracy, we created a new experimental database of organic redox-active molecules by the data from hundreds of published papers and assembled previous computational datasets. We also considered some key parameters, such as pH conditions and solvent type, to assess their impact on redox potential prediction. Our GPR model predicted redox potentials with high accuracy across all datasets using minimal training data. The study provides powerful tools for molecule screening and design and delivers valuable guidance on designing training datasets for costly experiments.

25 ENERGY STORAGE↗

Quantifying the Occurrence of Record Hot Years Through Normalized Warming Trends

Surface air temperature trends and extreme events are of global concern and they are related. Here we show that the occurrence of record hot years over different latitudes from 1960-2019 are more strongly correlated with the observational annual mean temperature trends normalized by internal variability. Compared with the raw trends showing Arctic amplification, the normalized trends show a tropical amplification over land. Two hot spots with more frequent occurrence of record hot years are identified: northern hemisphere ocean (versus land) and southern hemisphere tropical land (versus mid- and high-latitude lands). Ensemble mean results from 32 Earth system models agree with observations better than individual models, but they do not reproduce observed large differences in correlations across latitudes between normalized trends and record-breaking events over land versus ocean. Furthermore, our results enable the quantification of record hot year occurrence through normalized warming trends and provide new metrics for model evaluation and improvement.

58 GEOSCIENCES↗

Mississippi State University Center for Air Sea Technology FY95 Research Program

The Mississippi State University (MSU) Center for Air Sea Technology (CAST) evolved from the Institute for Naval Oceanography's (INO) Experimental Center for Mesoscale Ocean Prediction (ECMOP) which was started in 1989. MSU CAST subsequently began operation on 1 October 1992 under an Office of Naval Research (ONR) two-year grant which ended on 30 September 1994. In FY95 MSU CAST was successful in obtaining five additional research grants from ONR, as well as several other research contracts from the Naval Oceanographic Office via NASA, the Naval Research Laboratory, the Army Corps of Engineers, and private industry. In the past, MSU CAST technical research and development has produced tools, systems, techniques, and procedures that improve efficiency and overcome deficiency for both the operational and research communities residing with the Department of Defense, private industry, and university ocean modeling community. We continued this effort with the following thrust areas: to develop advanced methodologies and tools for model evaluation, validation and visualization, both oceanographic and atmospheric; to develop a system-level capability for conducting temporally and ; spatially scaled ocean simulations driven by or are responsive to ocean models, and take into consideration coupling to atmospheric models; to continue the existing oceanographic/atmospheric data management task with emphasis on distributed databases in a network environment, with database optimization and standardization, including use of Mosaic and World Wide Web (WWW) access; and to implement a high performance parallel computing technology for CAST ocean models

Yeske, Lanny↗

The Research of MOX Fuels in Japan

In Japan, uranium and plutonium mixed oxide (MOX) fuels has been developed for sodium-cooled fast reactors. The developing MOX fuels come in variety of O/M ratio, Pu content, minor actinide (MA) content and density. We have studied a science based fuel technology to evaluate fuel behaviors in fabrication process and irradiation condition of such various fuels. The technologies which are constructed based on experimental database can apply to mechanistic evaluation of fuel behaviors. To develop the science based fuel technology, many different varieties of basic properties have been investigated, and experimental database was constructed. And a mechanistic physical property model has been studied. The models contribute to describe various behaviors in fuel fabrication process and irradiation condition. In Japan, the researches on MOX fuel technologies have been carried out for sodium cooled fast reactor development. It is important to develop the fuel technologies on fuel fabrication process and irradiation performance model evaluation. The researches will be carried out by experimental approach and expanded in international collaboration frame works such as the CNWG and The Generation IV International Forum (GIF). (author)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Scalable Risk Assessment of Rare Events in Power Systems With Uncertain Wind Generation and Loads

Risk assessment of rare events has become increasingly important in power system planning and operation with the increasing integration of renewable energy and the presence of system uncertainties. However, quantifying the risk posed by rare events via the traditional method, i.e., Monte Carlo sampling (MCS), incurs substantial computational expense stemming from the vast ensemble of power flow simulations. To accelerate the assessment, this paper proposes a Deep Neural Network (DNN)-kernelized vector-valued Gaussian Process (VVGP) approach with excellent computational efficiency while maintaining high accuracy. Consequently, serving as a surrogate model for the power flow solver, the DNN-kernelized VVGP enables significantly faster but accurate risk assessment compared to the power flow solver. The developed surrogate model evaluates low-order N - k events that contain more than 90% instances by adeptly capturing the topological features while the high-order N - k events are assessed via a power flow solver, thereby striking a balance between computational efficiency and uncertainty quantification accuracy. Moreover, the model incorporates a Support Vector Machine (SVM) classifier to resample concerning low-probability tail events to counteract the biases potentially introduced during the DNN-kernelized VVGP evaluations. Simulations conducted on the modified IEEE 24-bus, 118-bus, and European 1354-bus systems demonstrate that the proposed method maintains the accuracy benchmark set by MCS while significantly reducing computational demands in large-scale power systems as compared to other state-of-the-art methods.

17 WIND ENERGY↗

A Physics-based Smart Persistence model for Intra-hour forecasting of solar radiation (PSPI) using GHI measurements and a cloud retrieval technique

Short-term solar forecasting models based solely on global horizontal irradiance (GHI) measurements are often unable to discriminate the forecasting of the factors affecting GHI from those that can be precisely computed by atmospheric models. Here, we introduce a Physics-based Smart Persistence model for Intra-hour forecasting of solar radiation (PSPI) that decomposes the forecasting of GHI into the computation of extraterrestrial solar radiation and solar zenith angle and the forecasting of cloud albedo and cloud fraction. The extraterrestrial solar radiation and solar zenith angle are accurately computed by the Solar Position Algorithm (SPA) developed at the National Renewable Energy Laboratory (NREL). A cloud retrieval technique is used to estimate cloud albedo and cloud fraction from surface-based observations of GHI. With the assumption of persistent cloud structures, the cloud albedo and cloud fraction are predicted for future time steps using a two-stream approximation and a 5-min exponential weighted moving average, respectively. Our model evaluation using the long-term observations of GHI at NREL's Solar Radiation Research Laboratory (SRRL) shows that the PSPI has a better performance than the persistence and smart persistence models in all forecast time horizons between 5 and 60 min, which is more significant in cloudy-sky conditions. Finally, compared to the persistence and smart persistence models, the PSPI does not require additional observations of various atmospheric parameters but is customizable in that additional observations, if available, can be ingested to further improve the GHI forecast. An advanced technology of cloud forecast is also expected to improve the future performance of the PSPI.

14 SOLAR ENERGY↗

Model Validation Database for Fires Involving Fuels at Liquefied Natural Gas Facilities

This document provides a description of the model evaluation protocol (MEP) database for fires involving liquefied natural gas (LNG) and processing fuels at LNG facilities. The purpose of the MEP is to provide procedures regarding the assessment of a model's suitability to predict thermal exclusion zones resulting from a fire. The database includes measurements from pool fire, jet fire, and fireball experiments which are provided in a spreadsheet. Users are to enter model results into the spreadsheet which automatically generates statistical performance measures and graphical comparisons with the experimental data. The intent of this document is to provide a description of the experiments and of the procedure required to carry out the validation portion of the MEP. In addition, the statistical performance measures, measurements for comparisons, and parameter variation are provided.

03 NATURAL GAS↗