All Explanations are Wrong but Some are Useful: Using Global Sensitivity Analysis to Identify Gaps in Machine Learning Explainability.
Abstract not provided.
Smith, Michael Reed↗
SEARCH · Engineering Papers
Results for “Global Sensitivity Analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Assessing the Fidelity of Explanations with Global Sensitivity Analysis.
Abstract not provided.
Smith, Michael↗
Global sensitivity analysis in Monte Carlo radiation transport
Explore the source record for details and available documents.
Clements, Kayla Beth↗
Uncertainty Quantification and Global Sensitivity Analysis Methods for Monte Carlo Radiation Transport Solvers
Explore the source record for details and available documents.
Clements, Kayla [Oregon State University]↗
BSM Sensitivity Analysis and Meta-Modeling Next Steps [Slides]
We performed a global sensitivity analysis of the Bioenergy Scenario Model (BSM), using Elementary Effects analysis to identify potentially influential factors, with a focus on sustainable aviation fuel (SAF) production. The process was iterative and informed key model changes and updates over the course of the study in addition to identifying the key model inputs that affect the cumulative production of SAF from the present through 2050. More work will be performed to gain deeper insight into the sensitivity of individual pathways as part of our larger efforts to develop a reduced-form version of the BSM.
09 BIOMASS FUELS↗
Physiological Controls on Carbon Fluxes and Biomass Production in Miscanthus: Insights From a Process‐Based Agroecosystem Model
Biomass crops serve as essential feedstocks for renewable energy and bioproducts and play a critical role in achieving lower emissions in the transportation sector. However, dedicated perennial biomass crops such as Miscanthus × giganteus (Miscanthus) remain underrepresented in process-based agroecosystem models, limiting robust evaluation of their economic and environmental performance. In this study, we developed a data-constrained representation of the sterile triploid Miscanthus (IL clone) within the process-based model ecosys, integrating global sensitivity analysis, ensemble simulation, and parameter calibration. Planting, harvesting, and fertilization practices consistent with field management were incorporated, and phenology was constrained using PhenoCam-derived Green Chromatic Coordinate (GCC) data. Using the Morris global sensitivity analysis method, we identified 11 key physiological parameters governing plant carbon, water, and nutrient relations, particularly processes associated with CO 2 assimilation. We then conducted ensemble simulations by perturbing these parameters and calibrated the model against eddy covariance fluxes and field-measured biomass. Building on the calibrated operating state, parameter-response analyses show that different photosynthetic processes influence productivity in different ways. Protein allocation determines whether productivity increases toward a higher level, whereas electron transport capacity controls additional gains once protein allocation approaches saturation. These findings demonstrate that parameter importance depends on physiological context and on which photosynthetic processes remain limiting. Calibration and validation against observations show that ecosys can reliably reproduce carbon and water fluxes, as well as both above- and belowground biomass, with post-calibration GPP R 2 improving from 0.67 to 0.95 during the calibration period and remaining high during validation (R 2 = 0.95). These results provide a mechanistic foundation for regional simulations and sustainable bioenergy assessments.
ecosys↗
Arctic Tipping Points Triggering Global Change (LDRD Final Report)
The Arctic is warming and feedbacks in the coupled Earth system may be driving the Arctic to tipping events that could have critical downstream impacts for the rest of the globe. In this project we have focused on analyzing sea ice variability and loss in the coupled Earth system Summer sea ice loss is happening rapidly and although the loss may be smooth and reversible, it has significant consequences for other Arctic systems as well as geopolitical and economic implications. Accurate seasonal predictions of sea ice minimum extent and long-term estimates of timing for a seasonally ice-free Arctic depend on a better understanding of the factors influencing sea ice dynamics and variation in this strongly coupled system. Under this project we have investigated the most influential factors in accurate predictions of September Arctic sea ice extent using machine learning models trained separately on observational data and on simulation data from five E3SM historical ensembles. Monthly averaged data from June, July, and August for a selection of ice, ocean, and atmosphere variables were used to train a random forest regression model. Gini importance measures were computed for each input feature with the testing data. We found that sea ice volume is most important earlier in the season (June) and sea ice extent became a more important predictor closer to September. Results from this study provide insight into how feature importance changes with forecast length and illustrates differences between observational data and simulated Earth system data. We have additionally performed a global sensitivity analysis (GSA) using a fully coupled ultra- low resolution configuration E3SM. To our knowledge, this is the first global sensitivity analysis involving the fully-coupled E3SM Earth system model. We have found that parameter variations show significant impact on the Arctic climate state and atmospheric parameters related to cloud parameterizations are the most significant. We also find significant interactions between parameters from different components of E3SM. The results of this study provide invaluable insight into the relative importance of various parameters from the sea ice, atmosphere and ocean components of the E3SM (including cross-component parameter interactions) on various Arctic-focused quantities of interest (QOIs).
54 ENVIRONMENTAL SCIENCES↗
Evaluating SWAT + model uncertainties for human and natural outcomes: Application in a Great Lakes agricultural watershed
Nutrient exports from agricultural lands in the Great Lakes Region pose significant threats to water quality and ecological health through eutrophication, hypoxia, and harmful algal blooms. Climate change and agricultural adaptation practices complicate future nutrient loading due to intensified hydrologic cycles and land use decisions. Our research focuses on evaluating the Soil and Water Assessment Tool (SWAT) plus model parametric uncertainties for human and natural outcomes across different scales. These factors are integral to ensuring a balance between productive agricultural practices and maintaining the health of watershed hydrology. However, uncertainties in modeling such complex interactions pose significant challenges, limiting our ability to precisely determine critical factors that influence crop yield and soil moisture. Our analysis employs Sobol global sensitivity analysis to evaluate first-order, second order, and total-order indices for SWAT crop growth parameter, ensuring comprehensive assessment of individual and interactive effects on model outputs. The objective is to identify the parameters that significantly affect model outputs for crop yield and soil moisture and improve our understanding of their interactions at the basin and hydrological response unit (HRU) scale. Our case study, the Portage River Watershed, which drains into Lake Erie, is chosen to better capture finer scale interactions crucial for predicting nutrient loading under future climate scenarios. This foundational work is aimed at setting the stage for the future development of an agent-based model (ABM). The ABM model would incorporate SWAT outputs to dynamically simulate decision-making processes.
Bunyon, Enock↗
Data for Physiological Controls on Carbon Fluxes and Biomass Production in Miscanthus: Insights From a Process- Based Agroecosystem Model
Biomass crops serve as essential feedstocks for renewable energy and bioproducts and play a critical role in achieving lower emissions in the transportation sector. However, dedicated perennial biomass crops such as Miscanthus × giganteus (Miscanthus) remain underrepresented in process- based agroecosystem models, limiting robust evaluation of their economic and environmental performance. In this study, we developed a data- constrained representation of the sterile triploid Miscanthus (IL clone) within the process- based model ecosys, integrating global sensitivity analysis, ensemble simulation, and parameter calibration. Planting, harvesting, and fertilization practices consistent with field management were incorporated, and phenology was constrained using PhenoCam- derived Green Chromatic Coordinate (GCC) data. Using the Morris global sensitivity analysis method, we identified 11 key physiological parameters governing plant carbon, water, and nutrient relations, particularly processes associated with CO2 assimilation. We then conducted ensemble simulations by perturbing these parameters and calibrated the model against eddy covariance fluxes and field- measured biomass. Building on the calibrated operating state, parameter- response analyses show that different photosynthetic processes influence productivity in different ways. Protein allocation determines whether productivity increases toward a higher level, whereas electron transport capacity controls additional gains once protein allocation approaches saturation. These findings demonstrate that parameter importance depends on physiological context and on which photosynthetic processes remain limiting. Calibration and validation against observations show that ecosys can reliably reproduce carbon and water fluxes, as well as both above- and belowground biomass, with post- calibration GPP R2 improving from 0.67 to 0.95 during the calibration period and remaining high during validation (R2 = 0.95). These results provide a mechanistic foundation for regional simulations and sustainable bioenergy assessments.
Carbon↗
Sensitivity analysis of generic deep geologic repository with focus on spatial heterogeneity induced by stochastic fracture network generation
Geologic Disposal Safety Assessment Framework is a state-of-the-art simulation software toolkit for probabilistic post-closure performance assessment of systems for deep geologic disposal of nuclear waste developed by the United States Department of Energy. This paper presents a generic reference case and shows how it is being used to develop and demonstrate performance assessment methods within the Geologic Disposal Safety Assessment Framework that mitigate some of the challenges posed by high uncertainty and limited computational resources. Variance-based global sensitivity analysis is applied to assess the effects of spatial heterogeneity using graph-based summary measures for scalar and time-varying quantities of interest. Behavior of the system with respect to spatial heterogeneity is further investigated using ratios of water fluxes. This analysis shows that spatial heterogeneity is a dominant uncertainty in predictions of repository performance which can be identified in global sensitivity analysis using proxy variables derived from graph descriptions of discrete fracture networks. New quantities of interest defined using water fluxes proved useful for better understanding overall system behavior.
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Quantifying the Uncertainty of the Future Hydrological Impacts of Climate Change: Comparative Analysis of an Advanced Hierarchical Sensitivity in Humid and Semiarid Basins
Comparison and quantification of different uncertainties of future climate change involved in the modeling of a hydrological system are highly important for both hydrological modelers and policy-makers. However, few studies have accurately estimated the relative importance of different sources of uncertainty at different spatiotemporal scales. Here, a hierarchical sensitivity analysis framework (HSAF) incorporated with a variance-based global sensitivity analysis is developed to quantify the spatiotemporal contributions of different uncertainties in hydrological impacts of climate change in two different climatic (humid and semiarid) basins in China. The uncertainty sources include three emission scenarios (ESs), 20 global climate models (GCs), three hydrological models (HMs), and the associated sensitive hydrological parameters (PAs) screened and sampled by the Morris and Latin hypercube sampling methods, respectively. Further, the results indicate that the overall trend of uncertainty is PA > HM > GC > ES, but their uncertainties have discrepancies in projections of different hydrological variables. The HM uncertainty in annual and monthly discharge projections is generally larger than the PA uncertainty in the humid basin than semiarid basin. The PA has greater uncertainty in extreme hydrological event (annual peak discharge) projections than in annual discharge projections for both basins (particularly for the humid basin), but contributes larger uncertainty to annual and monthly discharge projections in the semiarid basin than humid basin. The GC contributes larger uncertainty in all the hydrological variables projections in the humid basin than semiarid basin, while the ES uncertainty is rather limited in both basins. Overall, our results suggest there is greater spatiotemporal variability of hydrological uncertainty in more arid regions.
54 ENVIRONMENTAL SCIENCES↗
Role of the social factors in success of solar photovoltaic reuse and recycle programmes
By 2050, the cumulative mass of end-of-life photovoltaic (PV) modules may reach 80 Mt globally. The impacts could be mitigated by module recycling, repair and reuse; however, previous studies of PV circularity omit the consideration of critical social factors. Here we used an agent-based model to integrate social aspects with techno-economic factors, which provides a more realistic assessment of the circularity potential for previously studied interventions that assesses additional interventions that cannot be analysed using techno-economic analysis alone. We also performed a global sensitivity analysis using a machine-learning metamodel. We show that to exclude social factors underestimates the effect of lower recycling prices on PV material circularity, which highlights the relevance of considering social factors in future studies. Interventions aimed at changing customer attitudes about used PV boost the reuse of modules, although used modules can only satisfy one-third of the US demand during 2020–2050, which suggests that reuse should be complemented by recycling.
14 SOLAR ENERGY↗
Sensitivity Analysis for the Component Design App: Analysis of Success Assured Data
A new tool has been developed to perform variance-based global sensitivity analysis (VBGSA) on data from a set-based concurrent engineering software called Success Assured (SA). The tool is part of a digital component design app, which is currently in production as an Accelerated Digital Engineering Pathfinder at Sandia National Laboratories. When working with complex digital models, it is important to understand relationships between inputs and outputs, i.e., how “sensitive” model outputs are to changes in model inputs. After extensive research and trials of various sensitivity analysis methods, it was determined that estimation of Sobol’ indices for VBGSA with Monte Carlo simulation, paired with simple surrogate models, produces the best results for SA data. This tool increases understanding of SA models and streamlines the creation of SA datasets. This report details the methodology and implementation of this sensitivity analysis tool so others can understand it and implement it.
97 MATHEMATICS AND COMPUTING↗
Challenges and Lessons Learned in Applying Sensitivity Analysis to Building Stock Energy Models
Uncertainty Analysis (UA) and Sensitivity Analysis (SA) offer essential tools to determine the limits of inference of a model and explore the factors which have the most effect on the model outputs. However, despite a well established body of work applying UA and SA to models of individual buildings, a review of the literature relating to energy models for larger groups of buildings undertaken by Fennell et al. (2019) highlighted very limited application at larger scales. This contribution describes the efforts undertaken by a group of research teams in the context of IEA-EBC Annex 70 working with a diverse set of Building Stock Models (BSMs) to apply global sensitivity analysis methods and compare their results. Since BSMs are a class of model defined by their output and coverage rather than their structure and inputs, they represent a diverse set of modelling approaches. Key challenges for the application of SA are identified and explored, including the influence of model form, input data types and model outputs. This study combines results from 7 different modelling teams, each using different models across a range of urban areas to explore these challenges and begin the process of developing standardised workflows for SA of BSMs.
ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗
Assessment of model parameters in MFiX particle-in-cell approach
The limitations in numerical treatment of solids-phase in conventional methods like Discrete Element Model and Two-Fluid Model have facilitated the development of alternative techniques such as Particle-In-Cell (PIC). However, a number of parameters are involved in PIC due to its empiricism. In this work, global sensitivity analysis of PIC model parameters is performed under three distinct operating regimes common in chemical engineering applications, viz. settling bed, bubbling fluidized bed and circulating fluidized bed. Simulations were performed using the PIC method in Multiphase Flow with Interphase eXchanges (MFiX) developed by National Energy Technology Laboratory (NETL). A non-intrusive uncertainty quantification (UQ) based approach is applied using Nodeworks to first construct an adequate surrogate model and then identify the most influential parameters in each case. This knowledge will aid in developing an effective design of experiments and determine optimal parameters through techniques such as deterministic or statistical calibration.
01 COAL, LIGNITE, AND PEAT↗
Process-informed adsorbent design guidelines for direct air capture
Direct air capture using solid adsorbents is a proven technology critical to reducing our net greenhouse gas emissions to zero and beyond. Currently, academic research into the technology mainly focuses on the development of new adsorbents. However, there is a discord between the adsorbent design and process performance. Many materials scientists focus on maximising metrics such as the CO 2 capacity of their adsorbent. Here, we combine detailed process modelling, machine learning, and extensive global sensitivity analysis, which entails varying all of the model parameters together, on a direct air capture process to show that the dry CO 2 adsorption capacity does not influence process performance for an amine-functionalised adsorbent operating in a temperature vacuum swing adsorption (TVSA) process, while it is important in a steam-assisted TVSA (S-TVSA) process. In fact, adsorption kinetics, density, and thermal conductivity are all critical attributes to obtaining a low energy penalty and reduced costs. The analysis also highlights the importance of heat transfer, directing process engineers to (alternative) adsorber designs that maximise this. By an in-depth evaluation of how process performance indicators are affected by materials properties and process operating parameters, this work provides guidance to both material scientists and process engineers towards the design of a “unicorn adsorbent” and intensified DAC processes. This will improve the performance of solid adsorbent direct air capture and help drive down the costs of this vital technology to avert the worst impacts of climate change.
42 ENGINEERING↗
Polynomial Chaos Surrogate Construction for Random Fields with Parametric Uncertainty
Engineering and applied science rely on computational experiments to rigorously study physical systems. The mathematical models used to probe these systems are highly complex, and sampling-intensive studies often require prohibitively many simulations for acceptable accuracy. Surrogate models provide a means of circumventing the high computational expense of sampling such complex models. In particular, polynomial chaos expansions (PCEs) have been successfully used for uncertainty quantification studies of deterministic models where the dominant source of uncertainty is parametric. We discuss an extension to conventional PCE surrogate modeling to enable surrogate construction for stochastic computational models that have intrinsic noise in addition to parametric uncertainty. We develop a PCE surrogate on a joint space of intrinsic and parametric uncertainty, enabled by Rosenblatt transformations, which are evaluated via kernel density estimation of the associated conditional cumulative distributions. Furthermore, we extend the construction to random field data via the Karhunen–Loève expansion. We then take advantage of closed-form solutions for computing PCE Sobol indices to perform a global sensitivity analysis of the model which quantifies the intrinsic noise contribution to the overall model output variance. Additionally, the resulting joint PCE is generative in the sense that it allows generating random realizations at any input parameter setting that are statistically approximately equivalent to realizations from the underlying stochastic model. The method is demonstrated on a chemical catalysis example model and a synthetic example controlled by a parameter that enables a switch from unimodal to bimodal response distributions.
97 MATHEMATICS AND COMPUTING↗
Stochastic simulations of temporally and spatially variable Fe cycling within a floodplain aquifer
This data package contains model input and output files for local and global sensitivity analysis (SA) of a reactive transport model simulating redox cycling within a floodplain aquifer. The reaction network is implemented in CrunchFlow and focuses on spatially and temporally heterogeneous Fe cycling, though it contains several other species and redox pathways (37 aqueous species and 7 minerals in total). The package contains data from 600 Monte Carlo simulations (for global SA) and 104 one-at-a-time simulations (for local SA). It includes all input files required to run the CrunchFlow model, as well as limited model output – .csv files containing time series of primary species concentrations at 21 grid cells within the domain. These time series files contain the following species: H+, CO32-, SO42-, Cl-, Ca2+, Mg2+, Na+, K+, Fe(II), Fe(III), HS-, CH3COO-, O2(aq), S(aq), and Br-.These data were used to demonstrate the application of a novel form of global SA, distance-based generalized sensitivity analysis (DGSA) to a reactive transport model. The 2D model simulates the export of reduced species from pockets of fine-grained, organic-rich sediments embedded in a coarse sand aquifer. Stochastic simulations jointly varied 17 key model input parameters across several orders of magnitude, including the spatial variability of each parameter. DGSA on the model results reveal that the amplitude and variability of exported Fe(II) are most sensitive to the interaction between sand permeability and individual reaction rates. By contrast, the propagation of reducing conditions downgradient from the fine-grained lenses depends most heavily on the rate of dissolved organic carbon and sulfate release from the fines.
54 ENVIRONMENTAL SCIENCES↗