Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model diagnostics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Jacobian-based model diagnostics and application to equation oriented modeling of a carbon capture system

It can be difficult to identify the specific variables or equations responsible for convergence issues in large mathematical programming models. The Institute for the Design of Advanced Energy Systems Integrated Platform (IDAES-IP) contains a tool to identify poorly scaled constraints and variables by searching for rows and columns of the Jacobian matrix with small L2-norms. A singular value decomposition is then performed to identify degenerate sets of equations and remaining scaling issues. Here, this work presents a flowsheet developed for post-combustion carbon capture using a monoethanolamine (MEA) solvent system as a case study. This work takes the reader through the entire process of model diagnostics and reformulation, from a basic introduction to the mathematics behind these model diagnostics to the reformulations necessary to make the model numerically robust, including a significantly modified enhancement factor model.

IDAES↗
DOE OSTI2021

Integrated assessment model diagnostics: key indicators and model evolution

Integrated assessment models (IAMs) form a prime tool in informing climate mitigation strategies. Diagnostic indicators that allow to compare these models can help to describe and explain differences in model projections. This also increases transparency and comparability. Earlier, the IAM community has developed an approach to diagnose models (Kriegler et al., 2015). Here we build on this, by proposing a selected set of well-defined indicators as a community standard, similar to metrics used for other modeling communities such as climate models. These indicators are the relative abatement index (RAI), emission reduction type index (ERT), inertia timescale (IT), fossil fuel reduction (FFR), transformation index (TI) and cost per abatement value (CAV). We apply the approach to 17 IAMs, including both older version as well as their latest versions, as applied in the IPCC 6th Assessment Report (AR6). The study shows that the approach can be easily applied and allows for comparison of model versions in time. The indicators and their trends can often be explained in terms of model characteristics and changes. We show that together, the set of six indicators can provide an useful indication of the main traits of the model and can roughly indicate the general model behavior. The results also show that there is often a considerable spread across the models. Interestingly, the diagnostic values often change for different model versions, but there does not seem to be a distinct trend across the different models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Model Diagnostics for Equation Oriented Models: Roadblocks and the Path Forward

This poster was presented at the Foundations of Computer Aided Process Design (FOCAPD) 2024 conference with a paper describing efforts to develop a unified workflow and toolbox for diagnosing issues in equation-oriented models as part of the IDAES modeling framework. The poster presents the work currently underway within the IDAES project to develop and integrate cutting edge model analysis techniques into a common toolbox and workflow for model developers to use.

Lee, Andrew↗
DOE OSTI2024

Model Diagnostics for Equation Oriented Models: Roadblocks and the Path Forward

Conference paper for FOCAPD24 describing efforts to develop a unified workflow and toolbox for diagnosing issues in equation-oriented models as part of the IDAES modeling framework. The paper presents the work currently underway within the IDAES project to develop and integrate cutting edge model analysis techniques into a common toolbox and workflow for model developers to use.

Lee, Andrew↗
DOE OSTI2017

On the climate policy implications of substitutability and flexibility in the economy: An in-depth integrated assessment model diagnostic

This paper conducts an in-depth model diagnostic exercise for two parameters, 1) the elasticity of substitution between the capital/labour aggregate and the energy aggregate in the Integrated Assessment Model (IAM) MERGE's production function and 2) the rate at which new technologies can be deployed within the energy system. We show that in a more complementary world the model's ability to adjust the carbon intensity of its energy sector is more important whereas in a more substitutable world the ability to expand carbon free technologies is of lesser relative importance. The uncertainty in the literature surrounding the elasticity of substitution parameter, its interaction with the mechanisms of technical change, and the associated danger of grounding forward-looking analyses in historically based parameters lend support to the importance of such a diagnostic exercise. Building on work from model intercomparison studies, we investigate whether a given model's choice of strategy is primarily a function of the choice of its parameter values or its structure. As a result, a deeper understanding of what drives model behaviour is beneficial to both modellers and the policymakers who utilise their insights and output.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Jacobian-based Model Diagnostics and Application to Equation Oriented Modeling of a Carbon Capture System

Equation-oriented (EO) modeling has the potential to enable the effective design and optimization of the operation of advanced energy systems. However, advanced modeling of energy systems results in a large number of variables and non-linear equations, and it can be difficult to search through these to identify the culprit(s) responsible for convergence issues. The Institute for the Design of Advanced Energy Systems Integrated Platform (IDAES-IP) contains a tool to identify poorly scaled constraints and variables by searching for rows and columns of the Jacobian matrix with small L2-norms so they can be rescaled. A further singular value decomposition can be per-formed to identify degenerate sets of equations and remaining scaling issues. This work presents an EO model of a flowsheet developed for post-combustion carbon capture using a monoethanolamine (MEA) solvent system as a case study. The IDAES diagnostics tools were successfully applied to this flowsheet to identify problems to improve model robustness and enable the optimization of process design and operating conditions of a carbon capture system.

Allan, Douglas↗
DOE OSTI2021

A Bayesian Calibration Framework with Embedded Model Error for Model Diagnostics

We study the utility and performance of a Bayesian model error embedding construction in the context of molecular dynamics modeling of metallic alloys, where we embed model error terms in existing interatomic potential model parameters. To alleviate the computational burden of this approach, we propose a framework combining likelihood approximation and Gaussian process surrogates. Here we leverage sparse Gaussian process techniques to construct a hierarchy of increasingly accurate but more expensive surrogate models. This hierarchy is then exploited by multilevel Markov chain Monte Carlo methods to efficiently sample from the target posterior distribution. We illustrate the utility of this approach by calibrating an interatomic potential model for a family of gold-copper alloys. In particular, this case study highlights effective means for dealing with computational challenges with Bayesian model error embedding in large-scale physical models, and the utility of embedded model error for model diagnostics.

Bayesian inference↗
DOE OSTI2023

Rainbows and Climate Change: A tutorial on climate model diagnostics and parameterization

Earth System Models (ESMs) must represent processes below the grid scale of a model using representations (parameterizations) of physical and chemical processes. As a tutorial exercise to understand diagnostics and parameterization, this work presents a representation of rainbows for an ESM: the Community Earth System Model version 2 (CESM2). Using the 'state' of the model, basic physical laws, and some assumptions, we generate a representation of this unique optical phenomena as a diagnostic output. Rainbow occurrence and it's possible changes are related to cloud occurrence and rain formation which are critical uncertainties for climate change prediction. The work highlights issues which are typical of many diagnostics parameterizations such as assumptions, uncertain parameters and the difficulty of evaluation against uncertain observations. Results agree qualitatively with limited available global 'observations' of Rainbows. Rainbows are seen in expected locations in the sub-tropics over the ocean where broken clouds and frequent precipitation occurs. The diurnal peak is in the morning over ocean and in the evening over land. The representation of rainbows is found to be quantitatively sensitive to the assumed amount of cloudiness and the amount of stratiform rain. Rainbows are projected to have decreased, mostly in the Northern Hemisphere, due to aerosol pollution effects increasing cloud coverage since 1850. In the future, continued climate change is projected to decrease cloud cover, associated with a positive cloud feedback. As a result the rainbow diagnostic projects that rainbows will increase in the future, with the largest changes at mid-latitudes. The diagnostic may be useful for assessing cloud parameterizations, and is an exercise in how to build and test parameterizations of atmospheric phenomena.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Advancing Diagnostic Model Evaluation to Better Understand Water Shortage Mechanisms in Institutionally Complex River Basins

Abstract Water resources systems models enable valuable inferences on consequential system stressors by representing both the geophysical processes determining the movement of water and the human elements distributing it to its various competing uses. This study contributes a diagnostic evaluation framework that pairs exploratory modeling with global sensitivity analysis to enhance our ability to make inferences on water scarcity vulnerabilities in institutionally complex river basins. Diagnostic evaluation of models representing institutionally complex river basins with many stakeholders poses significant challenges. First, it needs to exploit a large and diverse suite of simulations to capture important human‐natural system interactions as well as institutionally aware behavioral mechanisms. Second, it needs to have performance metrics that are consequential and draw on decision‐relevant model outputs that adequately capture the multisector concerns that emerge from diverse basin stakeholders. We demonstrate the proposed model diagnostic framework by evaluating how potential interactions between changing hydrologic conditions and human demands influence the frequencies and durations of water shortages of varying magnitudes experienced by hundreds of users in a subbasin of the Colorado River. We show that the dominant factors shaping these effects vary both across users and, for an individual user, across percentiles of shortage magnitude. These differences hold even for users sharing diversion locations, demand levels or water right seniority. Our findings underline the importance of detailed institutional representation for such basins, as institutions strongly shape how dominant factors of stakeholder vulnerabilities propagate through the complex network of users.

Hadjimichael, Antonia↗
DOE OSTI2016

Hierarchical mixture of experts and diagnostic modeling approach to reduce hydrologic model structural uncertainty

In most water resources applications, a single model structure might be inadequate to capture the dynamic multi-scale interactions among different hydrological processes. Calibrating single models for dynamic catchments, where multiple dominant processes exist, can result in displacement of errors from structure to parameters, which in turn leads to over-correction and biased predictions. An alternative to a single model structure is to develop local expert structures that are effective in representing the dominant components of the hydrologic process and adaptively integrate them based on an indicator variable. In this study, the Hierarchical Mixture of Experts (HME) framework is applied to integrate expert model structures representing the different components of the hydrologic process. Various signature diagnostic analyses are used to assess the presence of multiple dominant processes and the adequacy of a single model, as well as to identify the structures of the expert models. The approaches are applied for two distinct catchments, the Guadalupe River (Texas) and the French Broad River (North Carolina) from the Model Parameter Estimation Experiment (MOPEX), using different structures of the HBV model. Furthermore, the results show that the HME approach has a better performance over the single model for the Guadalupe catchment, where multiple dominant processes are witnessed through diagnostic measures. Whereas, the diagnostics and aggregated performance measures prove that French Broad has a homogeneous catchment response, making the single model adequate to capture the response.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Diagnostic Models for Wind Turbine Gearbox Components Using SCADA Time Series Data: Preprint

This paper presents a scalable, autonomous framework for diagnostics of wind turbine gearbox failures using a multi-feature supervisory control and data acquisition data set spanning multiple years. Because of the size of the data set studied, all algorithms were constructed to be scalable using a Spark-Hadoop data framework. An unsupervised approach was used to detect significant failures based on predetermined criteria. The initial criteria selected were an abnormal spike in turbine component temperature followed by a turbine power off, which helps reduce the number of potential false alarms. To detect abnormal spikes in component temperature, a model was introduced to adjust the temperature data for effects caused by ambient temperature and normal temperature increases when loaded. This study evaluates methods for normalizing temperature sensor data to identify anomalies. The models that performed the best were a linear regression and a multivariate polynomial regression. The proposed process for finding failures has tunable parameters that can be adjusted to be more or less sensitive. The combination of sensor data normalization and application of these criteria is successful in finding turbine failures resulting in downtime. The proposed methods can operate without failure or maintenance logs and can be utilized for offline analysis of large high-resolution data sets.

17 WIND ENERGY↗
DOE OSTI2020

Convective Transition Statistics over Tropical Oceans for Climate Model Diagnostics: GCM Evaluation

To assess deep convective parameterizations in a variety of GCMs and examine the fast-time-scale convective transition, a set of statistics characterizing the pickup of precipitation as a function of column water vapor (CWV), PDFs and joint PDFs of CWV and precipitation, and the dependence of the moisture–precipitation relation on tropospheric temperature is evaluated using the hourly output of two versions of the GFDL Atmospheric Model, version 4 (AM4), NCAR CAM5 and superparameterized CAM (SPCAM). The 6-hourly output from the MJO Task Force (MJOTF)/GEWEX Atmospheric System Study (GASS) project is also analyzed. Contrasting statistics produced from individual models that primarily differ in representations of moist convection suggest that convective transition statistics can substantially distinguish differences in convective representation and its interaction with the large-scale flow, while models that differ only in spatial–temporal resolution, microphysics, or ocean–atmosphere coupling result in similar statistics. Most of the models simulate some version of the observed sharp increase in precipitation as CWV exceeds a critical value, as well as that convective onset occurs at higher CWV but at lower column RH as temperature increases. While some models quantitatively capture these observed features and associated probability distributions, considerable intermodel spread and departures from observations in various aspects of the precipitation–CWV relationship are noted. For instance, in many of the models, the transition from the low-CWV, nonprecipitating regime to the moist regime for CWV around and above critical is less abrupt than in observations. Additionally, some models overproduce drizzle at low CWV, and some require CWV higher than observed for strong precipitation. For many of the models, it is particularly challenging to simulate the probability distributions of CWV at high temperature.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Error field and magnetic diagnostic modeling for W7-X

The prediction, detection, and compensation of error fields for the W7-X device will play a key role in achieving a high beta (Β = 5%), steady state (30 minute pulse) operating regime utilizing the island divertor system [1]. Additionally, detection and control of the equilibrium magnetic structure in the scrape-off layer will be necessary in the long-pulse campaign as bootstrapcurrent evolution may result in poor edge magnetic structure [2]. An SVD analysis of the magnetic diagnostics set indicates an ability to measure the toroidal current and stored energy, while profile variations go undetected in the magnetic diagnostics. An additional set of magnetic diagnostics is proposed which improves the ability to constrain the equilibrium current and pressure profiles. However, even with the ability to accurately measure equilibrium parameters, the presence of error fields can modify both the plasma response and diverter magnetic field structures in unfavorable ways. Vacuum flux surface mapping experiments allow for direct measurement of these modifications to magnetic structure. The ability to conduct such an experiment is a unique feature of stellarators. The trim coils may then be used to forward model the effect of an applied n = 1 error field. This allows the determination of lower limits for the detection of error field amplitude and phase using flux surface mapping. *Research supported by the U.S. DOE under Contract No. DE-AC02-09CH11466 with Princeton University.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2018

NMR-based serum metabolomics study reveals a innovative diagnostic model for missed abortion

A missed abortion (MA) is an in-utero death of the embryo or fetus before the 20th week of gestation with retained products of conception. In order to discover novel biomarkers for MA, a {sup 1}H NMR spectroscopy-based metabolomics approach was applied to detect human MA serum metabolic profiles. Serum samples were obtained from patients with MA (n = 15) and healthy controls (n = 9) for study. The NOESYPR1D spectrum combined with multi-variate pattern recognition analysis was used to cluster the groups and establish a disease-specific metabolites phenotype. Principal component analysis (PCA) and orthogonal partial least-squares discriminant analysis (OPLS-DA) models were capable of distinguishing MA patients from healthy subjects. The results revealed that 24 metabolites altered in MA patients compared with the control population. Metabolomic pathway analysis demonstrated that alanine, aspartate and glutamate metabolism, citrate cycle (TCA cycle), taurine and hypotaurine metabolism were significantly altered in MA. The results indicated that serum NMR-based metabolomic profiling method is sensitive and specific enough to distinguish MA and from healthy controls, this method could be developed as a clinically useful diagnostic tool for MA. The finding from the MA serum metabolic profiling shed a new light on further understanding of MA disease mechanisms.

60 APPLIED LIFE SCIENCES↗