Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Model interpretation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35
All sourcesNASA NTRSDOE OSTI
NASA NTRS2014

How to Develop and Interpret a Credibility Assessment of Numerical Models for Human Research: NASA-STD-7009 Demystified

In the wake of the Columbia accident, the NASA-STD-7009 [1] credibility assessment was developed as a unifying platform to describe model credibility and the uncertainties in its modeling predictions. This standard is now being adapted by NASAs Human Research Program to cover a wide range of numerical models for human research. When used properly, the standard can improve the process of code development by encouraging the use of best practices. It can also give management more insight in making informed decisions through a better understanding of the models capabilities and limitations.To a newcomer, the abstractions presented in NASA-STD-7009 and the sheer volume of information that must be absorbed can be overwhelming. This talk is aimed at describing the credibility assessment, which is the heart of the standard, in plain terms. It will outline how to develop a credibility assessment under the standard. It will also show how to quickly interpret the graphs and tables that result from the assessment and how to drill down from the top-level view to the foundation of the assessment. Finally, it will highlight some of the resources that are available for further study.

numerical analysis↗
DOE OSTI2023

Data, scripts, and figures associated with a manuscript studying impact of climate and topography on post-fire vegetation recovery.

This data package is associated with the publication “Impact of Topography and Climate on Post-fire Vegetation Recovery Across Different Burn Severity and Land Cover Types through Machine Learning” submitted to Remote Sensing of Environment (Zahura et al. 2023). In this research, a machine learning algorithm, random forest (RF), was utilized to examine the impact of climate and topography on post-fire vegetation recovery. We used enhanced vegetation index (EVI) to examine varying burn severity and land cover types. The data package includes the input files for RF model training, outputs from model predictions and analysis, and python scripts to run the model, analyze the results to understand model performance and interpretability, and plot manuscript figures. This data package contains three folders (Data, Scripts, and Figures), a file-level metadata (FLMD) csv, and a data dictionary (dd) csv. Please see Postfire_recovery_flmd.csv for a list of all files contained in this data package and descriptions for each. The data dictionary (Postfire_recovery_dd.csv) describes the csv column headers. The “Data” folder provides all the inputs and outputs to train the RF model, evaluate performance, and interpret predictions. The “Scripts” folder contains python scripts and jupyter notebooks for model training and result analysis. The “Figures” folder includes the figures used in the manuscript in “.png” and “.jpg” format.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1986

Interpretation of rapid rises in hard X-rays and microwaves with the thermal conduction front model

Impulsive hard X-ray and microwave bursts with rise times from 0.1 to 10 seconds are discussed. Source areas calculated by the method of Crannell et al. (1978) were compared with source areas determined from Hinotori and the Hard X-ray Imaging Spectrometer (HXIS) images. The agreement strongly suggests that the method is valid. If the thermal conduction front model for the hard X-ray and microwave source is adopted, then the method enables derivation of area, density, magnetic field, and rise time from hard X-ray and microwave spectral observations. This approach was used to derive these parameters for several rapid impulsive rises in the flares of July 1, 1980, and May 21, 1984. It is shown that the model provides a consistent interpretation of the observations of these impulsive increases. Indeed, the model provides a way to calculate rise times from spectra alone (to within a factor of about three) over more than two orders of magnitude.

Batchelor, D. A.↗
DOE OSTI2019

Frio 2 : Raw CASSM Datasets

This submission contains the raw SEG-2 seismic datasets acquired in the CASSM experiment, part of the Frio-2 GCS pilot. The directory also includes the log files generated by the Geode seismograph (geometrics) used during acquisition. All SEG-2 files have a ".dat" suffix and log files have a ".log" suffix. Various supplementary files are also included (see below) including a set of flow and seismic models used in interpretation. The earlyGathers.tar.gz resource includes raw data acquired during system testing and pre-injection periods. All files correspond to the single source continuous active source seismic monitoring (CASSM) dataset described in the following papers. Daley, T.M., Solbau, R.D., Ajo-Franklin, J.B., and Benson, S.M. 2007, “Continuous Active-Source Seismic Monitoring of CO2 Injection in a Brine Aquifer”, Geophysics, Vol. 72, No. 5, p. A57-A61 Daley, T., Ajo-Franklin, J.B., and C. Doughty, 2011, “Constraining the reservoir model of an injected CO2 plume with crosswell CASSM at the Frio-II Brine Pilot”, International Journal of Greenhouse Gas Control (IJGGC), Vol. 5, No. 2, pp. 1022-1030. Zhu, T., Ajo-Franklin, J., and T. Daley, 2017, “Spatio-temporal changes in seismic attenuation caused by injected CO2 at the Frio-II pilot site, Dayton TX, USA”, Journal of Geophysical Research – Solid Earth, Vol. 122, No. 9, pp. 7156-7171. cassmDataRaw.tar : The main tar file containing all of the SEG-2 field records and the Geode log files. Stacking, time sampling (dt), and date information should be in the .log file. Each file corresponds to a single stacked gather for the single source deployed. Data was recorded on a 24 channel downhole hydrophone array. fieldNotes.tar : contains some of the scanned observer logs from the experiments as well as some planning images fieldPhotos.tar : contains a small set of photographs taken during the experiment frio_flow_seismic_models.mat : a matlab structure containing a set of flow and seismic models used in interpreting the CASSM data (see IJGGC paper).

GCS↗
DOE OSTI2023

Using machine learning with optical profilometry for GaN wafer screening

Abstract To improve the manufacturing process of GaN wafers, inexpensive wafer screening techniques are required to both provide feedback to the manufacturing process and prevent fabrication on low quality or defective wafers, thus reducing costs resulting from wasted processing effort. Many of the wafer scale characterization techniques—including optical profilometry—produce difficult to interpret results, while models using classical programming techniques require laborious translation of the human-generated data interpretation methodology. Alternatively, machine learning techniques are effective at producing such models if sufficient data is available. For this research project, we fabricated over 6000 vertical PiN GaN diodes across 10 wafers. Using low resolution wafer scale optical profilometry data taken before fabrication, we successfully trained four different machine learning models. All models predict device pass and fail with 70–75% accuracy, and the wafer yield can be predicted within 15% error on the majority of wafers.

36 MATERIALS SCIENCE↗
DOE OSTI2017

Three dimensional view of the SYK/AdS duality

We show that the spectrum of the SYK model can be interpreted as that of a 3D scalar coupled to gravity. The scalar has a mass which is at the Breitenholer-Freedman bound of AdS 2 , and subject to a delta function potential at the center of the interval along the third direction. This, through Kaluza-Klein procedure on AdS 2 × (S 1 )/Z 2 , generates the spectrum reproducing the bi-local propagator at strong coupling. Furthermore, the leading 1/J correction calculated in this picture reproduces the known correction to the poles of the SYK propagator, providing credence to a conjecture that the bulk dual of this model can be interpreted as a three dimensional theory.

2D Gravity↗
DOE OSTI2022

Global biomass supply modeling for long-run management of the climate system

Bioenergy is projected to have a prominent, valuable, and maybe essential, role in climate management. However, there is significant variation in projected bioenergy deployment results, as well as concerns about the potential environmental and social implications of supplying biomass. Bioenergy deployment projections are market equilibrium solutions from integrated modeling, yet little is known about the underlying modeling of the supply of biomass as a feedstock for energy use in these modeling frameworks. We undertake a novel diagnostic analysis with ten global models to elucidate, compare, and assess how biomass is supplied within the models used to inform long-run climate management. With experiments that isolate and reveal biomass supply modeling behavior and characteristics (costs, emissions, land use, market effects), we learn about biomass supply tendencies and differences. The insights provide a new level of modeling transparency and understanding of estimated global biomass supplies that informs evaluation of the potential for bioenergy in managing the climate and interpretation of integrated modeling. For each model, we characterize the potential distributions of global biomass supply across regions and feedstock types for increasing levels of quantity supplied, as well as some of the potential societal externalities of supplying biomass. We also evaluate the biomass supply implications of managing these externalities. Finally, we interpret biomass market results from integrated modeling in terms of our new understanding of biomass supply. Overall, we find little consensus between models on where biomass could be cost-effectively produced and the implications. We also reveal model specific biomass supply narratives, with results providing new insights into integrated modeling bioenergy outcomes and differences. The analysis finds that many integrated models are considering and managing emissions and land use externalities of supplying biomass and estimating that environmental and societal trade-offs in the form of land emissions, land conversion, and higher agricultural prices are cost-effective, and to some degree a reality of using biomass, to address climate change.

09 BIOMASS FUELS↗
DOE OSTI2023

A detailed study of interpretability of deep neural network based top taggers

Abstract Recent developments in the methods of explainable artificial intelligence (XAI) allow researchers to explore the inner workings of deep neural networks (DNNs), revealing crucial information about input–output relationships and realizing how data connects with machine learning models. In this paper we explore interpretability of DNN models designed to identify jets coming from top quark decay in high energy proton–proton collisions at the Large Hadron Collider. We review a subset of existing top tagger models and explore different quantitative methods to identify which features play the most important roles in identifying the top jets. We also investigate how and why feature importance varies across different XAI metrics, how correlations among features impact their explainability, and how latent space representations encode information as well as correlate with physically meaningful quantities. Our studies uncover some major pitfalls of existing XAI methods and illustrate how they can be overcome to obtain consistent and meaningful interpretation of these models. We additionally illustrate the activity of hidden layers as neural activation pattern diagrams and demonstrate how they can be used to understand how DNNs relay information across the layers and how this understanding can help to make such models significantly simpler by allowing effective model reoptimization and hyperparameter tuning. These studies not only facilitate a methodological approach to interpreting models but also unveil new insights about what these models learn. Incorporating these observations into augmented model design, we propose the particle flow interaction network model and demonstrate how interpretability-inspired model augmentation can improve top tagging performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Evidential Deep Learning: Enhancing Predictive Uncertainty Estimation for Earth System Science Applications

Abstract Robust quantification of predictive uncertainty is a critical addition needed for machine learning applied to weather and climate problems to improve the understanding of what is driving prediction sensitivity. Ensembles of machine learning models provide predictive uncertainty estimates in a conceptually simple way but require multiple models for training and prediction, increasing computational cost and latency. Parametric deep learning can estimate uncertainty with one model by predicting the parameters of a probability distribution but does not account for epistemic uncertainty. Evidential deep learning, a technique that extends parametric deep learning to higher-order distributions, can account for both aleatoric and epistemic uncertainties with one model. This study compares the uncertainty derived from evidential neural networks to that obtained from ensembles. Through applications of the classification of winter precipitation type and regression of surface-layer fluxes, we show evidential deep learning models attaining predictive accuracy rivaling standard methods while robustly quantifying both sources of uncertainty. We evaluate the uncertainty in terms of how well the predictions are calibrated and how well the uncertainty correlates with prediction error. Analyses of uncertainty in the context of the inputs reveal sensitivities to underlying meteorological processes, facilitating interpretation of the models. The conceptual simplicity, interpretability, and computational efficiency of evidential neural networks make them highly extensible, offering a promising approach for reliable and practical uncertainty quantification in Earth system science modeling. To encourage broader adoption of evidential deep learning, we have developed a new Python package, Machine Integration and Learning for Earth Systems (MILES) group Generalized Uncertainty for Earth System Science (GUESS) (MILES-GUESS) ( https://github.com/ai2es/miles-guess ), that enables users to train and evaluate both evidential and ensemble deep learning. Significance Statement This study demonstrates a new technique, evidential deep learning, for robust and computationally efficient uncertainty quantification in modeling the Earth system. The method integrates probabilistic principles into deep neural networks, enabling the estimation of both aleatoric uncertainty from noisy data and epistemic uncertainty from model limitations using a single model. Our analyses reveal how decomposing these uncertainties provides valuable insights into reliability, accuracy, and model shortcomings. We show that the approach can rival standard methods in classification and regression tasks within atmospheric science while offering practical advantages such as computational efficiency. With further advances, evidential networks have the potential to enhance risk assessment and decision-making across meteorology by improving uncertainty quantification, a longstanding challenge. This work establishes a strong foundation and motivation for the broader adoption of evidential learning, where properly quantifying uncertainties is critical yet lacking.

Schreck, John S.↗
DOE OSTI2026

Search for higgsinos in compressed mass spectra using low-momentum tracks in pp collisions at s=13 TeV with the ATLAS detector

This paper presents two searches for the electroweak production of higgsinos with compressed mass spectra using 140 fb−1 of s=13$$ \sqrt{s}=13 $$ TeV proton-proton collision data collected by the ATLAS experiment at the Large Hadron Collider. Events are required to feature an energetic jet, large missing transverse momentum, and at least one low-momentum charged particle that serves as a candidate higgsino decay product. In the first search, targeting higgsino mass splittings in the range of 0.3–1 GeV, the higgsinos are expected to predominantly decay into pions that are identified as low-momentum charged particles with large transverse impact parameters due to the long higgsino lifetime (cτ ≈ ?(0.1–10 mm)), and neural networks are used to discriminate between signal and background processes. The second search targets larger mass splittings in the range of 1–3 GeV, where the higgsinos are expected to decay promptly into low-momentum leptons, one of which is identified by dedicated low-momentum electron or muon taggers based on neural networks utilising tracking and calorimeter information. No significant excess above the Standard Model prediction is observed in either search and the results are interpreted within simplified models, to set lower limits on the masses of the higgsino-like charginos and neutralinos. Together, these searches exclude chargino masses below 126 GeV at 95% confidence level for mass splittings between the chargino and lightest neutralino in the range of 0.3–2 GeV. This represents the first ATLAS constraints in a portion of this parameter space and surpasses the limits previously set by other experiments.

Aad, G↗
NASA NTRS1984

Remote Sensing of Earth Terrain

Work performed on remote sensing of each terrain is outlined. The purpose of the investigation was: (1) development of theoretical models for remote sensing of Earth terrain; (2) data analysis and interpretation using the models developed; (3) scene simulation; and (4) development of new remote sensing approaches and techniques.

Kong, J. A.↗
NASA NTRS2012

Scaling of Lift Degradation Due to Anti-Icing Fluids Based Upon the Aerodynamic Acceptance Test

In recent years, the FAA has worked with Transport Canada, National Research Council Canada (NRC) and APS Aviation, Inc. to develop allowance times for aircraft operations in ice-pellet precipitation. These allowance times are critical to ensure safety and efficient operation of commercial and cargo flights. Wind-tunnel testing with uncontaminated anti-icing fluids and fluids contaminated with simulated ice pellets had been carried out at the NRC Propulsion and Icing Wind Tunnel (PIWT) to better understand the flowoff characteristics and resulting aerodynamic effects. The percent lift loss on the thin, high-performance wing model tested in the PIWT was determined at 8 angle of attack and used as one of the evaluation criteria in determining the allowance times. Because it was unclear as to how performance degradations measured on this model were relevant to an actual airplane configuration, some means of interpreting the wing model lift loss was deemed necessary. This paper describes how the lift loss was related to the loss in maximum lift of a Boeing 737-200ADV airplane through the Aerodynamic Acceptance Test (AAT) performed for fluids qualification. A loss in maximum lift coefficient of 5.24 percent on the B737-200ADV airplane (which was adopted as the threshold in the AAT) corresponds to a lift loss of 7.3 percent on the PIWT model at 8 angle of attack. There is significant scatter in the data used to develop the correlation related to varying effects of the anti-icing fluids that were tested and other factors. A statistical analysis indicated the upper limit of lift loss on the PIWT model was 9.2 percent. Therefore, for cases resulting in PIWT model lift loss from 7.3 to 9.2 percent, extra scrutiny of the visual observations is required in evaluating fluid performance with contamination.

Broeren, Andy P.↗
NASA NTRS2012

Review of the Aerodynamic Acceptance Test and Application to Anti-Icing Fluids Testing in the NRC Propulsion and Icing Wind Tunnel

In recent years, the FAA has worked with Transport Canada, National Research Council of Canada (NRC) and APS Aviation, Inc. to develop allowance times for aircraft operations in ice-pellet precipitation. These allowance times are critical to ensure safety and efficient operation of commercial and cargo flights. Wind-tunnel testing with uncontaminated anti-icing fluids and fluids contaminated with simulated ice-pellets had been carried out at the NRC Propulsion and Icing Wind Tunnel (PIWT) to better understand the flowoff characteristics and resulting aerodynamic effects. The percent lift loss on the thin, high-performance wing model tested in the PIWT was determined at 8 angle of attack and used as one of the evaluation criteria in determining the allowance times. Because it was unclear as to how performance degradations measured on this model were relevant to an actual airplane configuration, some means of interpreting the wing model lift loss was deemed necessary. In this report, the lift loss was related to the loss in maximum lift of a Boeing 737-200ADV airplane through the Aerodynamic Acceptance Test (AAT) performed for fluids qualification. This report provides a review of the research basis of the AAT in order to understand how this correlation was applied. A loss in maximum lift coefficient of 5.24 percent on the B737-200ADV airplane (which was adopted as the threshold in the AAT) corresponds to a lift loss of 7.3 percent on the PIWT model at 8 degrees angle of attack. There is significant scatter in the data used to develop the correlation related to varying effects of the various antiicing fluids that were tested and other factors. A statistical analysis indicated the upper limit of lift loss on the PIWT model was 9.2 percent. Therefore, for cases resulting in PIWT model lift loss from 7.3 to 9.2 percent, extra scrutiny of the visual observations is required in evaluating fluid performance with contamination. Additional research may result in future changes to this correlation.

Broeren, Andy P.↗
NASA NTRS2012

Scaling of Lift Degradation Due to Anti-Icing Fluids Based Upon the Aerodynamic Acceptance Test

In recent years, the FAA has worked with Transport Canada, National Research Council Canada (NRC) and APS Aviation, Inc. to develop allowance times for aircraft operations in ice-pellet precipitation. These allowance times are critical to ensure safety and efficient operation of commercial and cargo flights. Wind-tunnel testing with uncontaminated anti-icing fluids and fluids contaminated with simulated ice pellets had been carried out at the NRC Propulsion and Icing Wind Tunnel (PIWT) to better understand the flow-off characteristics and resulting aerodynamic effects. The percent lift loss on the thin, high-performance wing model tested in the PIWT was determined at 8 deg. angle of attack and used as one of the evaluation criteria in determining the allowance times. Because it was unclear as to how performance degradations measured on this model were relevant to an actual airplane configuration, some means of interpreting the wing model lift loss was deemed necessary. This paper describes how the lift loss was related to the loss in maximum lift of a Boeing 737-200ADV airplane through the Aerodynamic Acceptance Test (AAT) performed for fluids qualification. A loss in maximum lift coefficient of 5.24% on the B737-200ADV airplane (which was adopted as the threshold in the AAT) corresponds to a lift loss of 7.3% on the PIWT model at 8 deg. angle of attack. There is significant scatter in the data used to develop the correlation related to varying effects of the anti-icing fluids that were tested and other factors. A statistical analysis indicated the upper limit of lift loss on the PIWT model was 9.2%. Therefore, for cases resulting in PIWT model lift loss from 7.3% to 9.2%, extra scrutiny of the visual observations is required in evaluating fluid performance with contamination.

Broeren, Andy↗
DOE OSTI2018

Modeling sequential context effects in diagnostic interpretation of screening mammograms

Prior research has shown that physicians’ medical decisions can be influenced by sequential context, particularly in cases where successive stimuli exhibit similar characteristics when analyzing medical images. This type of systematic error is known to psychophysicists as sequential context effect as it indicates that judgments are influenced by features of and decisions about the preceding case in the sequence of examined cases, rather than being based solely on the peculiarities unique to the present case. We determine if radiologists experience some form of context bias, using screening mammography as the use case. To this end, we explore correlations between previous perceptual behavior and diagnostic decisions and current decisions. We hypothesize that a radiologist’s visual search pattern and diagnostic decisions in previous cases are predictive of the radiologist’s current diagnostic decisions. To test our hypothesis, we tasked 10 radiologists of varied experience to conduct blind reviews of 100 four-view screening mammograms. Eye-tracking data and diagnostic decisions were collected from each radiologist under conditions mimicking clinical practice. Perceptual behavior was quantified using the fractal dimension of gaze scanpath, which was computed using the Minkowski–Bouligand box-counting method. To test the effect of previous behavior and decisions, we conducted a multifactor fixed-effects ANOVA. Further, to examine the predictive value of previous perceptual behavior and decisions, we trained and evaluated a predictive model for radiologists’ current diagnostic decisions. ANOVA tests showed that previous visual behavior, characterized by fractal analysis, previous diagnostic decisions, and image characteristics of previous cases are significant predictors of current diagnostic decisions. Additionally, predictive modeling of diagnostic decisions showed an overall improvement in prediction error when the model is trained on additional information about previous perceptual behavior and diagnostic decisions.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Interpretable machine learning for knowledge generation in heterogeneous catalysis

Most applications of machine learning in heterogeneous catalysis thus far have used black-box models to predict computable physical properties (descriptors), such as adsorption or formation energies, that can be related to catalytic performance (that is, activity or stability). Here, extracting meaningful physical insights from these black-box models has proved challenging, as the internal logic of these black-box models is not readily interpretable due to their high degree of complexity. Interpretable machine learning methods that merge the predictive capacity of black-box models with the physical interpretability of physics-based models offer an alternative to black-box models. In this Perspective, we discuss the various interpretable machine learning methods available to catalysis researchers, highlight the potential of interpretable machine learning to accelerate hypothesis formation and knowledge generation, and outline critical challenges and opportunities for interpretable machine learning in heterogeneous catalysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2004

Complexity of life via collective mind

e mind is introduced as a set of simple intelligent units (say, neurons, or interacting agents), which can communicate by exchange of information without explicit global control. Incomplete information is compensated by a sequence of random guesses symmetrically distributed around expectations with prescribed variances. Both the expectations and variances are the invariants characterizing the whole class of agents. These invariants are stored as parameters of the collective mind, while they contribute into dynamical formalism of the agents' evolution, and in particular, into the reflective chains of their nested abstract images of the selves and non-selves. The proposed model consists of the system of stochastic differential equations in the Langevin form representing the motor dynamics, and the corresponding Fokker-Planck equation representing the mental dynamics (Motor dynamics describes the motion in physical space, while mental dynamics simulates the evolution of initial errors in terms of the probability density). The main departure of this model from Newtonian and statistical physics is due to a feedback from the mental to the motor dynamics which makes the Fokker-Planck equation nonlinear. Interpretation of this model from mathematical and physical viewpoints, as well as possible interpretation from biological, psychological, and social viewpoints are discussed. The model is illustrated by the dynamics of a dialog.

collective intelligence↗
DOE OSTI2022

Higgs Portal Vector Dark Matter Interpretation: Review of Effective Field Theory Approach and Ultraviolet Complete Models

In this report a review of the Higgs portal-vector dark matter interpretation of the spin-independent dark matter nu-cleon elastic scattering cross section is presented, where the invisible Higgs decay width measured at the LHC is used. Effective Field Theory and ultraviolet complete models are discussed. LHC interpretations show only the scalar and Majorana dark matter scenarios; we propose including interpretation for vec-tor dark matter in the EFT and UV completion theoretical framework. In addition, our studies suggest an extension of the LHC dark matter interpretations to the sub-GeV regime.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗