Does a reduced model reproduce the uncertainty of the original full-size model?
Not Available
Liu, Zhenyuan (ORCID:0000000195777784)↗
SEARCH · Engineering Papers
Results for “Model uncertainty”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
pnnl/SNAP
In this work, we detail two uncertainty quantification (UQ) methods that provide complementary information. Readout ensembling, by finetuning only the readout layers of an ensemble of foundation models, provides information about model uncertainty. Amending the final readout layer to predict upper and lower quantiles replaces point predictions with distributional predictions, which provide information about uncertainty within the underlying training data. We demonstrate our approach with the MACE-MP-0 model, applying UQ to both the foundation model and a series of finetuned models. The uncertainties produced by the ensemble and quantile methods are demonstrated to be distinct measures by which the quality of the NNP output can be judged.
Pope, Jenna (Bilbrey) [Pacific Northwest National ↗
YAHFC: A Code Framework to Model Nuclear Reactions and Estimate Correlated Uncertainties
Reaction modeling is a key ingredient in designing experiments and interpreting their results, and is an essential component in the process of evaluating nuclear data and assembling nuclear data libraries used in nuclear technology applications. Typically, experimental data are available only for a handful of reaction channels and theory models are used to fill in the gaps. In addition, theory is often called upon as the arbitrator between discrepant data. Most importantly, theory and modeling are required for an accurate determination of uncertainties in the evaluated data and the correlations between the multiple channels. A fast, accurate, and flexible modeling capability has been developed at LLNL with the code system YAHFC (Yet Another Hauser-Feshbach Code). YAHFC is a Monte Carlo, Hauser-Feshbach code framework, making full use of dynamic memory allocation, derived types, and parallel computing. YAHFC can generate events to simulate experiments and is guiding experiments designed to measure inelastic neutron scattering from actinide targets. YAHFC is also being used to analyze decays from surrogate experiments, thereby enabling the inference of reaction cross sections inaccessible by direct measurement. Finally, by modeling nuclear reactions with constraints from experimental data, YAHFC can deliver complete nuclear data libraries, with evaluated uncertainties, using the modernized Generalized Nuclear Data Structure (GNDS).
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
A Novel Framework to Project the Permafrost Fate With Explicit Quantification of Soil Property and Future Climate Uncertainties
This study develops a novel general framework to project the permafrost fate with rigorous uncertainty quantification to assess dominant sources. Borehole temperature records from three sites in the Russian western Arctic are used to constrain the uncertainty of a high‐fidelity freeze‐thaw model. Projections from 9 Global Climate Models (GCM) are stochastically downscaled to generate future trajectories of surface ground heat flux. Under the two emission scenarios SSP2‐4.5 and SSP5‐8.5, the projected average thawing depths by 2100 vary from 0.4 to 14.4 m or 2.1 to 17.7 m, and the increase in the top 10 m average temperature from 2015 to 2100 is 1.2–2.7°C or 1.9–3.0°C. The results show that the freeze‐thaw model uncertainty can sometimes dominate over that of GCM outputs, calling for site‐specific information to improve model accuracy. The framework is applicable for understanding permafrost degradation and related uncertainties at larger scales.
Bayesian downscaling↗
Uncertainty in Thermal Modeling of Spent Nuclear Fuel Casks
Uncertainty is a key metric in computational modeling that must be evaluated for results to have wide ranging applicability. A well characterized uncertainty range is ideal with clear error bars on results that can be presented to stakeholders. In the field of spent fuel cask modeling, this ideal has been historically difficult to achieve in practice because of the computationally intensive nature of the models used and the difficulty assigning reasonable uncertainties to quantities in as-built systems. The work in this report has been conducted to evaluate the overall state of uncertainty and sensitivity in spent fuel cask models and develop methodologies for evaluating these uncertainties. These methodologies must be practical for engineering applications. They should not require excessive computational resources or calendar time to achieve results. In engineering, the model must be on a scale such that it can be changed and adapted throughout a project as new information is discovered and project goals evolve. This report covers three major modeling task areas that provide an overview of the types of sensitivity and uncertainty present in a spent fuel storage and transportation system. Section 3 discusses sensitivity and uncertainty analysis in the effective thermal conductivity model for the fuel region and applies these results to a single assembly model. Section 4 shows sensitivity analysis of a full cask model in the TN-32B and Section 5 demonstrates the overall uncertainty workflow using Coolant Boiling in Rod Arrays – Spent Fuel Storage and STAR-CCM+ developed from the sensitivity work in the preceding sections.
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Accelerated Carbon and Water Cycles in the Amazon and Congo Basins Revealed From TRENDY Models and Remote Sensing Products
Tropical forests play a vital role in the global carbon cycle and land–atmosphere interactions. Estimating tropical forest carbon–water dynamics is challenging due to observational and modeling uncertainties. This study leverages the “Trends and drivers of the regional scale terrestrial sources and sinks of carbon dioxide” (TRENDY) project models and satellite observations to assess changes (2003–2021) in vegetation carbon, gross primary production (GPP), evapotranspiration (ET), and net biosphere production (NBP) in the Amazon and Congo. Atmospheric CO 2 , climate variability, and land use and land cover changes constrain these variables between 1700 and 2021 with the overall increasing trends of carbon stock and fluxes. The models overestimate vegetation carbon and GPP, while ET and NBP are consistent with observations. Fire-activated models predict lower values for vegetation carbon and GPP, ET, and NBP, aligning more closely with observations. The higher ET from fire-activated models may result from enhanced soil evaporation due to increased canopy openings. Fire-inactivated models could well estimate the magnitudes of NBP. The high vegetation carbon in nitrogen-enabled models points to simulation uncertainties and imbalance in model numbers regarding the nitrogen cycle. Although the nitrogen cycle enhances water use efficiency in both the Amazon and Congo, the models show a higher sensitivity to the nitrogen cycle in the Congo. This study highlights the challenges in accurately representing tropical biogeochemical cycles and the values of satellite products in model evaluations, underscoring the need for standard modeling protocols that address biogeochemical components (e.g., nutrient cycles) to better resolve process-based representations.
Shi, Mingjie [Pacific Northwest National Laborator↗
Benchmarking FFTF LOFWOS Test# 13 using SAM code: Baseline model development and uncertainty quantification
The development and deployment of advanced reactors, such as the sodium-cooled fast reactor (SFR), relies on sophisticated modeling tools to ensure the safety of the design under various transients. The predictive capability of these advanced modeling tools requires validation to garner trust in supporting the licensing of the advanced reactors. For this reason, the International Atomic Energy Agency (IAEA) initiated a coordinated research project (CRP) in 2018 for the analysis of the Fast Flux Test Facility (FFTF) Loss of Flow Without Scram (LOFWOS) Test #13.In this study, we present and discuss the benchmarking efforts of the modern system code SAM on the FFTF LOFWOS Test #13. Further, the SAM baseline model was developed according to the benchmark specification, which included a detailed core model with reactivity feedback. Generally, good agreement was observed between the baseline results and benchmark measurements; however, discrepancies persisted, particularly in predicted fuel assembly coolant outlet temperatures. Utilizing the baseline model, uncertainty quantification (UQ) and sensitivity analysis (SA) were conducted with the assistance of various statistical learning and machine learning methods, including kernel density estimation, Gaussian processes, and Sobol indices. Following the baseline model prediction and UQ and SA results, we discuss the reasons for the simulation discrepancies and propose further improvements to the model. This benchmarking effort adheres to the best-estimate plus uncertainty approach and can serve as a valuable example for supporting risk-informed licensing of advanced reactors.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Cherenkov Telescope Array sensitivity to the putative millisecond pulsar population responsible for the Galactic Centre excess
ABSTRACT The leading explanation of the Fermi Galactic Centre γ-ray excess is the extended emission from an unresolved population of millisecond pulsars (MSPs) in the Galactic bulge. Such a population would, along with the prompt γ-rays, also inject large quantities of electrons/positrons (e±) into the interstellar medium. These e± could potentially inverse-Compton (IC) scatter ambient photons into γ-rays that fall within the sensitivity range of the upcoming Cherenkov Telescope Array (CTA). In this article, we examine the detection potential of CTA to this signature by making a realistic estimation of the systematic uncertainties on the Galactic diffuse emission model at TeV-scale γ-ray energies. We forecast that, in the event that e± injection spectra are harder than E−2, CTA has the potential to robustly discover the IC signature of a putative Galactic bulge MSP population sufficient to explain the Galactic Centre excess for e± injection efficiencies in the range of ≈2.9–74.1 per cent, or higher, depending on the level of mismodelling of the Galactic diffuse emission components. On the other hand, for spectra softer than E−2.5, a reliable CTA detection would require an unphysically large e± injection efficiency of ${\gtrsim} 158{{\ \rm per\ cent}}$. However, even this pessimistic conclusion may be avoided in the plausible event that MSP observational and/or modelling uncertainties can be reduced. We further find that, in the event that an IC signal were detected, CTA can successfully discriminate between an MSP and a dark matter origin for the radiating e±.
79 ASTRONOMY AND ASTROPHYSICS↗
Cosmology from weak lensing, galaxy clustering, CMB lensing, and tSZ – I. 10 × 2pt modelling methodology
ABSTRACT The overlap of galaxy surveys and cosmic microwave background (CMB) experiments presents an ideal opportunity for joint cosmological data set analyses. In this paper we develop a halo model-based method for the first joint analysis combining these two experiments using 10 correlated two-point functions (10 × 2pt) derived from galaxy position, galaxy shear, CMB lensing convergence, and Compton-y fields. We explore this method using the Vera Rubin Observatory Legacy Survey of Space and Time (LSST) and the Simons Observatory (SO) as examples. We find such LSS × CMB joint analyses lead to significant improvement in Figure-of-Merit of Ωm and S8 over the constraints from using LSS-only probes within Λ cold dark matter (ΛCDM) model. We identify that the shear–y and y–y correlations are the most valuable additions when thermal Sunyaev–Zel’dolvich (tSZ) is included. We further identify the dominant sources of halo model uncertainties in the small-scale modelling, and investigate the impact of halo self-calibration due to the inclusion of small-scale tSZ information.
79 ASTRONOMY AND ASTROPHYSICS↗
Front-end engineering design (FEED) studies: a quantitative analysis
NETL has devised a methodology for normalizing FEED study metrics of interest and allowing cautious quantitative comparison across FEED studies. This presentation introduces the novel quantitative comparison methodology and presents results from utilizing this methodology to examine data presented in recent FEED study reports. The quantitative methodology developed for the examination of FEED study performance and cost also allows comparison of real-world performance and costs against NETL TEA model predicted performance and cost. Learnings from examining NETL model predicted performance and cost versus real world reported values are highlighted. These learnings provide insight into NETL TEA model uncertainty and highlight opportunities for further model development.
FEED Studies↗
Parameter estimation with maximal updated densities
A recently developed measure-theoretic framework solves a stochastic inverse problem (SIP) for models where uncertainties in model output data are predominantly due to aleatoric (i.e., irreducible) uncertainties in model inputs (i.e., parameters). The subsequent inferential target is a distribution on parameters. Another type of inverse problem is to quantify uncertainties in estimates of “true” parameter values under the assumption that such uncertainties should be reduced as more data are incorporated into the problem, i.e., the uncertainty is considered epistemic. A major contribution of this work is the formulation and solution of such a parameter identification problem (PIP) within the measure-theoretic framework developed for the SIP. The approach is novel in that it utilizes a solution to a stochastic forward problem (SFP) to update an initial density only in the parameter directions informed by the model output data. In other words, this method performs “selective regularization” only in the parameter directions not informed by data. Further, the solution is defined by a maximal updated density (MUD) point where the updated density defines the measure-theoretic solution to the PIP. Another significant contribution of this work is the full theory of existence and uniqueness of MUD points for linear maps with Gaussian distributions. Data-constructed Quantity of Interest (QoI) maps are also presented and analyzed for solving the PIP within this measure-theoretic framework as a means of reducing uncertainties in the MUD estimate. We conclude with a demonstration of the general applicability of the method on two problems involving either spatial or temporal data for estimating uncertain model parameters. The first problem utilizes spatial data from a stationary partial differential equation to produce a MUD estimate of an uncertain boundary condition. The second problem utilizes temporal data obtained from the state-of-the-art ADvanced CIRCulation (ADCIRC) model to obtain a MUD estimate of uncertain wind drag coefficients for a simulated extreme weather event near the Shinnecock Inlet located in the Outer Barrier of Long Island, NY, USA.
42 ENGINEERING↗
Confronting the water potential information gap
Water potential directly controls the function of leaves, roots and microbes, and gradients in water potential drive water flows throughout the soil–plant–atmosphere continuum. Notwithstanding its clear relevance for many ecosystem processes, soil water potential is rarely measured in situ, and plant water potential observations are generally discrete, sparse, and not yet aggregated into accessible databases. These gaps limit our conceptual understanding of biophysical responses to moisture stress and inject large uncertainty into hydrologic and land-surface models. Here, we outline the conceptual and predictive gains that could be made with more continuous and discoverable observations of water potential in soils and plants. We discuss improvements to sensor technologies that facilitate in situ characterization of water potential, as well as strategies for building new networks that aggregate water potential data across sites. Here, we end by highlighting novel opportunities for linking more representative site-level observations of water potential to remotely sensed proxies. Together, these considerations offer a road map for clearer links between ecohydrological processes and the water potential gradients that have the ‘potential’ to substantially reduce conceptual and modelling uncertainties.
58 GEOSCIENCES↗
Multifidelity Monte Carlo estimation for efficient uncertainty quantification in climate-related modeling
Abstract. Uncertainties in an output of interest that depends on the solution of a complex system (e.g., of partial differential equations with random inputs) are often, if not nearly ubiquitously, determined in practice using Monte Carlo (MC) estimation. While simple to implement, MC estimation fails to provide reliable information about statistical quantities (such as the expected value of the output of interest) in application settings such as climate modeling, for which obtaining a single realization of the output of interest is a costly endeavor. Specifically, the dilemma encountered is that many samples of the output of interest have to be collected in order to obtain an MC estimator that has sufficient accuracy – so many, in fact, that the available computational budget is not large enough to effect the number of samples needed. To circumvent this dilemma, we consider using multifidelity Monte Carlo (MFMC) estimation which leverages the use of less costly and less accurate surrogate models (such as coarser grids, reduced-order models, simplified physics, and/or interpolants) to achieve, for the same computational budget, higher accuracy compared to that obtained by an MC estimator – or, looking at it another way, an MFMC estimator obtains the same accuracy as the MC estimator at lower computational cost. The key to the efficacy of MFMC estimation is the fact that most of the required computational budget is loaded onto the less costly surrogate models so that very few samples are taken of the more expensive model of interest. We first provide a more detailed discussion about the need to consider an alternative to MC estimation for uncertainty quantification. Subsequently, we present a review, in an abstract setting, of the MFMC approach along with its application to three climate-related benchmark problems as a proof-of-concept exercise.
54 ENVIRONMENTAL SCIENCES↗
Active meta-learning for predicting and selecting perovskite crystallization experiments
Autonomous experimentation systems use algorithms and data from prior experiments to select and perform new experiments in order to meet a specified objective. In most experimental chemistry situations, there is a limited set of prior historical data available, and acquiring new data may be expensive and time consuming, which places constraints on machine learning methods. Active learning methods prioritize new experiment selection by using machine learning model uncertainty and predicted outcomes. Meta-learning methods attempt to construct models that can learn quickly with a limited set of data for a new task. Here in this paper, we applied the model-agnostic meta-learning (MAML) model and the Probabilistic LATent model for Incorporating Priors and Uncertainty in few-Shot learning (PLATIPUS) approach, which extends MAML to active learning, to the problem of halide perovskite growth by inverse temperature crystallization. Using a dataset of 1870 reactions conducted using 19 different organoammonium lead iodide systems, we determined the optimal strategies for incorporating historical data into active and meta-learning models to predict reaction compositions that result in crystals. We then evaluated the best three algorithms (PLATIPUS and active-learning k-nearest neighbor and decision tree algorithms) with four new chemical systems in experimental laboratory tests. With a fixed budget of 20 experiments, PLATIPUS makes superior predictions of reaction outcomes compared to other active-learning algorithms and a random baseline.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Global Sensitivity Analysis of a Reactive Transport Model for Mineral Scale Formation During Hydraulic Fracturing
Injection of water-based hydraulic fracturing fluid (HFF) into tight shale gas/oil formations can increase formation permeability and enhance production rates, but this process frequently causes mineral scale formation that can occlude pore space and hinder flow. To identify the most important factors that control the formation of mineral scales, we applied a novel global sensitivity analysis method—distance-based generalized sensitivity analysis (DGSA)—to a reactive transport model (RTM) that was previously built and calibrated to simulate precipitation of barite [BaSO4] and iron (hydr)oxide [Fe(OH) 3 ] in shale matrices and on fracture surfaces. Reactive transport simulations were run with model parameters randomly sampled based on assigned uncertainties. Modeling results for barite and Fe(OH)3 formation were clustered using machine-learning algorithms. A list of ranked critical input parameters was obtained after statistical quantification of cumulative distribution functions of input parameters. We found that barite formation is most sensitive to the rate of sulfate ion generation, which is determined by the pyrite dissolution rate coefficient and oxidant availability. In addition, barite formation is sensitive to the initial amounts of barite in HFF and shale, followed by barite thermodynamics/kinetics. For Fe(OH) 3 formation, the ranked factors are Fe(OH)3 precipitation rate coefficients, initial HFF pH, initial Fe(OH) 3 amount in HFF, and oxidant availability. Overall, our results provide insights into managing mineral scale formation during hydraulic fracturing to enhance production. Meanwhile, this study serves as an example of global sensitivity analysis of RTMs using the efficient, straightforward, and open-source DGSA method.
58 GEOSCIENCES↗
A tutorial review of machine learning-based model predictive control methods
Abstract This tutorial review provides a comprehensive overview of machine learning (ML)-based model predictive control (MPC) methods, covering both theoretical and practical aspects. It provides a theoretical analysis of closed-loop stability based on the generalization error of ML models and addresses practical challenges such as data scarcity, data quality, the curse of dimensionality, model uncertainty, computational efficiency, and safety from both modeling and control perspectives. The application of these methods is demonstrated using a nonlinear chemical process example, with open-source code available on GitHub. The paper concludes with a discussion on future research directions in ML-based MPC.
Wu, Zhe [Department of Chemical and Biomolecular E↗
Overview of Algorithms for Using Particle Morphology in Pre-Detonation Nuclear Forensics
A major goal in pre-detonation nuclear forensics is to infer the processing conditions and/or facility type that produced radiological material. This review paper focuses on analyses of particle size, shape, texture (“morphology”) signatures that could provide information on the provenance of interdicted materials. For example, uranium ore concentrates (UOC or yellowcake) include ammonium diuranate (ADU), ammonium uranyl carbonate (AUC), sodium diuranate (SDU), magnesium diuranate (MDU), and others, each prepared using different salts to precipitate U from solution. Once precipitated, UOCs are often dried and calcined to remove adsorbed water. The products can be allowed to react further, forming uranium oxides UO3, U3O8, or UO2 powders, whose surface morphology can be indicative of precipitation and/or calcination conditions used in their production. This review paper describes statistical issues and approaches in using quantitative analyses of measurements such as particle size and shape to infer production conditions. Statistical topics include multivariate t tests (Hotelling’s T 2 ), design of experiments, and several machine learning (ML) options including decision trees, learning vector quantization neural networks, mixture discriminant analysis, and approximate Bayesian computation (ABC). ABC is emphasized as an attractive option to include the effects of model uncertainty in the selected and fitted forward model used for inferring processing conditions.
98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗