Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “uncertainty Quantification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Uncertainty-aware mixed-variable machine learning for materials design

Abstract Data-driven design shows the promise of accelerating materials discovery but is challenging due to the prohibitive cost of searching the vast design space of chemistry, structure, and synthesis methods. Bayesian optimization (BO) employs uncertainty-aware machine learning models to select promising designs to evaluate, hence reducing the cost. However, BO with mixed numerical and categorical variables, which is of particular interest in materials design, has not been well studied. In this work, we survey frequentist and Bayesian approaches to uncertainty quantification of machine learning with mixed variables. We then conduct a systematic comparative study of their performances in BO using a popular representative model from each group, the random forest-based Lolo model (frequentist) and the latent variable Gaussian process model (Bayesian). We examine the efficacy of the two models in the optimization of mathematical functions, as well as properties of structural and functional materials, where we observe performance differences as related to problem dimensionality and complexity. By investigating the machine learning models’ predictive and uncertainty estimation capabilities, we provide interpretations of the observed performance differences. Our results provide practical guidance on choosing between frequentist and Bayesian uncertainty-aware machine learning models for mixed-variable BO in materials design.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Deep-learning-enabled Bayesian inference of fuel magnetization in magnetized liner inertial fusion

Fuel magnetization in magneto-inertial fusion (MIF) experiments improves charged burn product confinement, reducing requirements on fuel areal density and pressure to achieve self-heating. By elongating the path length of 1.01 MeV tritons produced in a pure deuterium fusion plasma, magnetization enhances the probability for deuterium–tritium reactions producing 11.8-17.1 MeV neutrons. Nuclear diagnostics thus enable a sensitive probe of magnetization. Characterization of magnetization, including uncertainty quantification, is crucial for understanding the physics governing target performance in MIF platforms, such as magnetized liner inertial fusion (MagLIF) experiments conducted at Sandia National Laboratories, Z-facility. We demonstrate a deep-learned surrogate of a physics-based model of nuclear measurements. A single model evaluation is reduced from O ( 10 – 100 ) (10–100) CPU hours on a high-performance computing cluster down to O ( 10 – 100 ) (10) ms on a laptop. This enables a Bayesian inference of magnetization, rigorously accounting for uncertainties from surrogate modeling and noisy nuclear measurements. The approach is validated by testing on synthetic data and comparing with a previous study. We analyze a series of MagLIF experiments systematically varying preheat, resulting in the first ever systematic experimental study of magnetic confinement properties of the fuel plasma as a function of fundamental inputs on any neutron-producing MIF platform. We demonstrate that magnetization decreases from B R ∼ 0.2 ~0.5 to B R ∼ 0.2 ~0.2 MG cm as laser preheat energy deposited increases from E preheat ∼ 460 ~460 J to E preheat ∼ 460 ~1.4 kJ. This trend is consistent with 2D LASNEX simulations showing Nernst advection of the magnetic field out of the hot fuel and diffusion into the target liner.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Model-free estimation of completeness, uncertainties, and outliers in atomistic machine learning using information theory

Abstract An accurate description of information is relevant for a range of problems in atomistic machine learning (ML), such as crafting training sets, performing uncertainty quantification (UQ), or extracting physical insights from large datasets. However, atomistic ML often relies on unsupervised learning or model predictions to analyze information contents from simulation or training data. Here, we introduce a theoretical framework that provides a rigorous, model-free tool to quantify information contents in atomistic simulations. We demonstrate that the information entropy of a distribution of atom-centered environments explains known heuristics in ML potential developments, from training set sizes to dataset optimality. Using this tool, we propose a model-free UQ method that reliably predicts epistemic uncertainty and detects out-of-distribution samples, including rare events in systems such as nucleation. This method provides a general tool for data-driven atomistic modeling and combines efforts in ML, simulations, and physical explainability.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Preliminary Results for Using Uncertainty and Out-of-distribution Detection to Identify Unreliable Predictions.

As machine learning (ML) models are deployed into an ever-diversifying set of application spaces, ranging from self-driving cars to cybersecurity to climate modeling, the need to carefully evaluate model credibility becomes increasingly important. Uncertainty quantification (UQ) provides important information about the ability of a learned model to make sound predictions, often with respect to individual test cases. However, most UQ methods for ML are themselves data-driven and therefore susceptible to the same knowledge gaps as the models themselves. Specifically, UQ helps to identify points near decision boundaries where the models fit the data poorly, yet predictions can score as certain for points that are under-represented by the training data and thus out-of-distribution (OOD). One method for evaluating the quality of both ML models and their associated uncertainty estimates is out-of-distribution detection (OODD). We combine OODD with UQ to provide insights into the reliability of the individual predictions made by an ML model.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Meeting Global Health Needs via Infectious Disease Forecasting: Development of a Reliable Data-Driven Framework

Infectious diseases (IDs) have a significant detrimental impact on global health. Timely and accurate ID forecasting can result in more informed implementation of control measures and prevention policies. To meet the operational decision-making needs of real-world circumstances, we aimed to build a standardized, reliable, and trustworthy ID forecasting pipeline and visualization dashboard that is generalizable across a wide range of modeling techniques, IDs, and global locations. We forecasted 6 diverse, zoonotic diseases (brucellosis, campylobacteriosis, Middle East respiratory syndrome, Q fever, tick-borne encephalitis, and tularemia) across 4 continents and 8 countries. We included a wide range of statistical, machine learning, and deep learning models (n=9) and trained them on a multitude of features (average n=2326) within the One Health landscape, including demography, landscape, climate, and socioeconomic factors. The pipeline and dashboard were created in consideration of crucial operational metrics—prediction accuracy, computational efficiency, spatiotemporal generalizability, uncertainty quantification, and interpretability—which are essential to strategic data-driven decisions. While no single best model was suitable for all disease, region, and country combinations, our ensemble technique selects the best-performing model for each given scenario to achieve the closest prediction. For new or emerging diseases in a region, the ensemble model can predict how the disease may behave in the new region using a pretrained model from a similar region with a history of that disease. The data visualization dashboard provides a clean interface of important analytical metrics, such as ID temporal patterns, forecasts, prediction uncertainties, and model feature importance across all geographic locations and disease combinations. As the need for real-time, operational ID forecasting capabilities increases, this standardized and automated platform for data collection, analysis, and reporting is a major step forward in enabling evidence-based public health decisions and policies for the prevention and mitigation of future ID outbreaks.

60 APPLIED LIFE SCIENCES↗
NASA NTRSDate not supplied

Interval Predictor Models for Robust System Identification

This paper proposes a framework for the identification and uncertainty quantification of plant models according to multivariable data. The only restriction imposed upon such models is for their outputs to depend continuously on their parameters. An Interval Predictor Model (IPM) prescribes the parameters of a computational model as a path-connected set thereby making each predicted output an interval-valued function of its inputs. The formulation proposed seeks the parameter set for which the predicted outputs tightly enclose the data. This set, which is modeled as a semi-algebraic set of low-degree polynomials, enables the characterization of possibly strong parameter dependencies commonly found in practice. This uncertainty characterization makes the resulting plant model amenable to robust control approaches using polynomial optimization. Furthermore, we use non-convex scenario theory to assess the reliability of the resulting IPM. This assessment yields a distribution-free upper bound on the probability that future data will fall outside the predicted intervals.

interval↗
DOE OSTI2020

Application of Neutron Multiplicity Counting Experiments to Optimal Cross-Section Adjustments

This paper presents the first application of model calibration to neutron multiplicity counting (NMC) experiments for cross-section optimization that is informed by adjoint-based sensitivity analysis (SA) and first-order uncertainty quantification (UQ). We summarize previous work on SA applied to NMC and describe notable modifications and additions. We give the procedure for first-order UQ and Bayesian-inference-based parameter estimation (PE). We then discuss model calibration applied to NMC of a 4.5-kg sphere of weapons-grade, alpha-phase plutonium metal (the BeRP ball) with the nPod neutron multiplicity counter. For the BeRP ball in bare and polyethylene-reflected configurations, we discuss the sensitivity of the first- and second-moment detector responses (i.e., first and second moments of the NMC distribution, respectively) to the cross sections. We describe the sources of uncertainty in the measured and simulated responses. Specifically, uncertainty in the measured responses is due to both random and systematic sources. Uncertainty in the simulated responses is due to the cross-section covariances. We describe in detail the adjustment to the cross sections and cross-section covariances due to the optimization. Due to the contribution of systematic uncertainties to the measured response uncertainties, the adjustment to the cross sections is similar in trend but larger in magnitude compared to that recommended by previous work. We compare the measured responses to responses simulated with nominal and optimized cross sections, demonstrating that the best-estimate cross sections produce simulations of NMC experiments that are more accurate with reduced uncertainty.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Chiral uncertainties in ab initio elastic nucleon-nucleus scattering

The effective interaction between a nucleon and a nucleus is one of the most important ingredients for reaction theories. Theoretical formulations were introduced early by Feshbach and Watson, and efforts of deriving and computing those ‘optical potentials’ in a microscopic fashion have a long tradition. However, only recently the leading order term in the Watson multiple scattering approach could be calculated fully ab initio, meaning that the same nucleon-nucleon (NN) interaction enters both the structure as well as the reaction pieces on equal footing. This allows the uncertainties from the underlying chiral effective NN interaction to be systematically explored in nucleon-nucleus elastic scattering observables. In this contribution the main ingredients for arriving at the ab initio leading order of the effective nucleon-nucleus interaction in the Watson approach will be reviewed. Concentrating on one specific chiral NN interaction from the LENPIC collaboration and light nuclei with a 0 + ground state, the leading order nucleon-nucleus interaction is calculated using up to the third chiral order (N2LO) in the nucleon-nucleon potential, and elastic scattering observables are extracted. Then pointwise as well as correlated uncertainty quantification is used for the estimation of the chiral truncation error. Elastic scattering observables for 4 He, 12 C, and 16 O for between 65 and 200 MeV projectile energy will be analyzed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Discrete-Direct Model Calibration and Uncertainty Propagation Method Confirmed on Multi-Parameter Plasticity Model Calibrated to Sparse Random Field Data

A discrete direct (DD) model calibration and uncertainty propagation approach is explained and demonstrated on a 4-parameter Johnson-Cook (J-C) strain-rate dependent material strength model for an aluminum alloy. The methodology's performance is characterized in many trials involving four random realizations of strain-rate dependent material-test data curves per trial, drawn from a large synthetic population. The J-C model is calibrated to particular combinations of the data curves to obtain calibration parameter sets which are then propagated to “Can Crush” structural model predictions to produce samples of predicted response variability. These are processed with appropriate sparse-sample uncertainty quantification (UQ) methods to estimate various statistics of response with an appropriate level of conservatism. This is tested on 16 output quantities (von Mises stresses and equivalent plastic strains) and it is shown that important statistics of the true variabilities of the 16 quantities are bounded with a high success rate that is reasonably predictable and controllable. The DD approach has several advantages over other calibration-UQ approaches like Bayesian inference for capturing and utilizing the information obtained from typically small numbers of replicate experiments in model calibration situations—especially when sparse replicate functional data are involved like force–displacement curves from material tests. The DD methodology is straightforward and efficient for calibration and propagation problems involving aleatory and epistemic uncertainties in calibration experiments, models, and procedures.

42 ENGINEERING↗
NASA NTRSDate not supplied

Uncertainty Estimates for Sonic-Boom Pressure Signatures and Loudness Carpets

A non-intrusive uncertainty quantification method is applied to computational analysis of supersonic, low-boom aircraft. The mean and standard deviation statistics of the pressure waveforms and loudness metrics are evaluated through use of numerical quadrature. The probability density function (p.d.f.) of these outputs is evaluated via kernel density estimation. The simulations use an inviscid, embedded-boundary Cartesian-mesh flow solver in the nearfield combined with an augmented Burgers’ equation solver for propagation in the farfield. The results show that the p.d.f. of the waveform is bimodal at shocks, which makes the mean and standard deviation statistics inappropriate. Despite this limitation, we show that the moment statistics can provide effective assessment of discrepancies when comparing with experimental data. This is demonstrated by presenting uncertainty analysis of a wind-tunnel test and showing that we significantly improve the predictions when we include the test uncertainties in the simulation. Normal distributions are obtained for the ground signature and loudness metrics, which is primarily due to the careful shaping of the low-boom waveform. Separation of variables and error control are used to reduce computational cost. We demonstrate that this is an efficient approach in the sense of balancing numerical errors in the statistics quadrature with discretization errors in the solvers.

ARMD↗
DOE OSTI2025

Robust Solution Verification Experiments on Nonuniform Meshes

The activities of verification, validation, and uncertainty quantification (VVUQ) provide a comprehensive means to assess the credibility of computational models. Within VVUQ, solution verification assesses numerical errors and evaluates whether the simulation is sufficiently accurate for its intended applications. As computational modeling gains traction in the development of complex, high-consequence systems, the need for robust solution verification intensifies, particularly because experimental data for these systems are often limited. This work examines improvements in the robustness of Richardson extrapolation (RE), a method commonly used in solution verification to study the discretization error of computational models using a power law. Nonuniform mesh refinement is discussed alongside other pollutants that affect the robustness of the power law model. Maximum likelihood estimation (MLE) is proposed as a robust strategy to address the uncertainty generated by nonuniform mesh refinement. An exploratory computational fluid dynamics (CFD) study of a 2D planar Poiseuille flow is conducted to determine if nonuniform mesh noise can be modeled with this MLE approach for more robust RE.

Weinmeister, Justin [ORNL] (ORCID:0000000160090237↗
DOE OSTI2021

Statistical emulation of a perturbed basal melt ensemble of an ice sheet model to better quantify Antarctic sea level rise uncertainties

Abstract. Antarctic ice shelves are vulnerable to warming ocean temperatures, and some have already begun thinning in response to increased basal melt rates. Sea level is therefore expected to rise due to Antarctic contributions, but uncertainties in its amount and timing remain largely unquantified. In particular, there is substantial uncertainty in future basal melt rates arising from multi-model differences in thermal forcing and how melt rates depend on that thermal forcing. To facilitate uncertainty quantification in sea level rise projections, we build, validate, and demonstrate projections from a computationally efficient statistical emulator of a high-resolution (4 km) Antarctic ice sheet model, the Community Ice Sheet Model version 2.1. The emulator is trained to a large (500-member) ensemble of 200-year-long 4 km resolution transient ice sheet simulations, whereby regional basal melt rates are perturbed by idealized (yet physically informed) trajectories. The main advantage of our emulation approach is that by sampling a wide range of possible basal melt trajectories, the emulator can be used to (1) produce probabilistic sea level rise projections over much larger Monte Carlo ensembles than are possible by direct numerical simulation alone, thereby providing better statistical characterization of uncertainties, and (2) predict the simulated ice sheet response under differing assumptions about basal melt characteristics as new oceanographic studies are published, without having to run additional numerical ice sheet simulations. As a proof of concept, we propagate uncertainties about future basal melt rate trajectories, derived from regional ocean models, to generate probabilistic sea level rise estimates for 100 and 200 years into the future.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Matilda v1.0: An R package for probabilistic climate projections using a reduced complexity climate model

A primary advantage to using reduced complexity climate models (RCMs) has been their ability to quickly conduct probabilistic climate projections, a key component of uncertainty quantification in many impact studies and multisector systems. Providing frameworks for such analyses has been a target of several RCMs used in studies of the future co-evolution of the human and Earth systems. In this paper, we present Matilda, an open-science R software package that facilitates probabilistic climate projection analysis, implemented here using the Hector simple climate model in a seamless and easily applied framework. The primary goal of Matilda is to provide the user with a turn-key method to build parameter sets from literature-based prior distributions, run Hector iteratively to produce perturbed parameter ensembles (PPEs), weight ensembles for realism against observed historical climate data, and compute probabilistic projections for different climate variables. This workflow gives the user the ability to explore viable parameter space and propagate uncertainty to model ensembles with just a few lines of code. The package provides significant freedom to select different scoring criteria and algorithms to weight ensemble members, as well as the flexibility to implement custom criteria. Additionally, the architecture of the package simplifies the process of building and analyzing PPEs without requiring significant programming expertise, to accommodate diverse use cases. We present a case study that provides illustrative results of a probabilistic analysis of mean global surface temperature as an example of the software application.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Seasonal variability of wake impacts on US mid-Atlantic offshore wind plant power production

Abstract. The mid-Atlantic will experience rapid wind plant development due to its promising wind resource located near large population centers. Wind turbines and wind plants create wakes, or regions of reduced wind speed, that may negatively affect downwind turbines and plants. We evaluate wake variability and annual energy production with the first yearlong modeling assessment using the Weather Research and Forecasting model, deploying 12 MW turbines across the domain at a density of 3.14 MW km−2, matching the planned density of 3 MW km−2. Using a series of simulations with no wind plants, one wind plant, and complete build-out of lease areas, we calculate wake effects and distinguish the effect of wakes generated internally within one plant from those generated externally between plants. We also provide a first step towards uncertainty quantification by testing the amount of added turbulence kinetic energy (TKE) by 0 % and 100 %. We provide a sensitivity analysis by additionally comparing 25 % and 50 % for a short case study period. The strongest wakes, propagating 55 km, occur in summertime stable stratification, just when New England's grid demand peaks in summer. The seasonal variability of wakes in this offshore region is much stronger than the diurnal variability of wakes. Overall, yearlong simulated wake impacts reduce power output by a range between 38.2 % and 34.1 % (for 0 %–100 % added TKE). Internal wakes cause greater yearlong power losses, from 29.2 % to 25.7 %, compared to external wakes, from 14.7 % to 13.4 %. The overall impact is different from the linear sum of internal wakes and external wakes due to non-linear processes. Additional simulations quantify wake uncertainty by modifying the added amount of turbulent kinetic energy from wind turbines, introducing power output variability of 3.8 %. Finally, we compare annual energy production to New England grid demand and find that the lease areas can supply 58.8 % to 61.2 % of annual load. We note that the results of this assessment are not intended to make nor are they suitable to make commercial judgments about specific wind projects.

17 WIND ENERGY↗
DOE OSTI2024

Elastic scattering of 3 He + 4 He with the SONIK scattering chamber

Measurements of the elastic scattering cross section of 3 He and 4 He are important in order to improve constraints on theoretical models of 4 He ⁢( 3 He ,𝛾)⁢ 7 Be, a key reaction in Big Bang nucleosynthesis and solar neutrino production. The astrophysical 𝑆 factor for this reaction is a significant source of uncertainty in the standard-solar-model prediction of the 7 Be and 8 B solar neutrino fluxes. The elastic scattering measurements reported in the literature do not extend to low energies and lack proper uncertainty quantification. A new measurement of the 4 He ⁢( 3 He, 3 He)⁢ 4 He reaction has been made at center-of-mass energies 𝐸c.m.=0.38−3.13 MeV using the Scattering of Nuclei in Inverse Kinematics (SONIK) scattering chamber: a windowless, extended gas target surrounded by an array of 30 collimated silicon charged particle detectors situated at TRIUMF. This is the first elastic scattering measurement of 3 He + 4 He made below 500 keV and it has greater angular range and better precision than previous measurements. The elastic scattering data were analyzed using both 𝑅-matrix and halo effective field theory frameworks, and values of the 𝑠-wave scattering length and effective range were extracted. Here, the resulting improvement in knowledge of the 𝑠-wave effective-range function at low energies reduces the overall uncertainty in 𝑆 34 at solar energies.

Elastic scattering reactions↗
NASA NTRS2010

Finite Element Model Calibration Approach for Ares I-X

Ares I-X is a pathfinder vehicle concept under development by NASA to demonstrate a new class of launch vehicles. Although this vehicle is essentially a shell of what the Ares I vehicle will be, efforts are underway to model and calibrate the analytical models before its maiden flight. Work reported in this document will summarize the model calibration approach used including uncertainty quantification of vehicle responses and the use of nonconventional boundary conditions during component testing. Since finite element modeling is the primary modeling tool, the calibration process uses these models, often developed by different groups, to assess model deficiencies and to update parameters to reconcile test with predictions. Data for two major component tests and the flight vehicle are presented along with the calibration results. For calibration, sensitivity analysis is conducted using Analysis of Variance (ANOVA). To reduce the computational burden associated with ANOVA calculations, response surface models are used in lieu of computationally intensive finite element solutions. From the sensitivity studies, parameter importance is assessed as a function of frequency. In addition, the work presents an approach to evaluate the probability that a parameter set exists to reconcile test with analysis. Comparisons of pre-test predictions of frequency response uncertainty bounds with measured data, results from the variance-based sensitivity analysis, and results from component test models with calibrated boundary stiffness models are all presented.

Horta, Lucas G.↗
NASA NTRS2010

Finite Element Model Calibration Approach for Area I-X

Ares I-X is a pathfinder vehicle concept under development by NASA to demonstrate a new class of launch vehicles. Although this vehicle is essentially a shell of what the Ares I vehicle will be, efforts are underway to model and calibrate the analytical models before its maiden flight. Work reported in this document will summarize the model calibration approach used including uncertainty quantification of vehicle responses and the use of non-conventional boundary conditions during component testing. Since finite element modeling is the primary modeling tool, the calibration process uses these models, often developed by different groups, to assess model deficiencies and to update parameters to reconcile test with predictions. Data for two major component tests and the flight vehicle are presented along with the calibration results. For calibration, sensitivity analysis is conducted using Analysis of Variance (ANOVA). To reduce the computational burden associated with ANOVA calculations, response surface models are used in lieu of computationally intensive finite element solutions. From the sensitivity studies, parameter importance is assessed as a function of frequency. In addition, the work presents an approach to evaluate the probability that a parameter set exists to reconcile test with analysis. Comparisons of pretest predictions of frequency response uncertainty bounds with measured data, results from the variance-based sensitivity analysis, and results from component test models with calibrated boundary stiffness models are all presented.

Horta, Lucas G.↗
DOE OSTI2020

Quantifying uncertainties and correlations in the nuclear-matter equation of state

We perform statistically rigorous uncertainty quantification (UQ) for chiral effective field theory (χ EFT) applied to infinite nuclear matter up to twice nuclear saturation density. The equation of state (EOS) is based on high-order many-body perturbation theory calculations with nucleon-nucleon and three-nucleon interactions up to fourth order in the χ EFT expansion. From these calculations our newly developed Bayesian machine-learning approach extracts the size and smoothness properties of the correlated EFT truncation error. Furthermore, we then propose a novel extension that uses multitask machine learning to reveal correlations between the EOS at different proton fractions. The inferred in-medium χ EFT breakdown scale in pure neutron matter and symmetric nuclear matter is consistent with that from free-space nucleon-nucleon scattering. These significant advances allow us to provide posterior distributions for the nuclear saturation point and propagate theoretical uncertainties to derived quantities: the pressure and incompressibility of symmetric nuclear matter, the nuclear symmetry energy, and its derivative. Our results, which are validated by statistical diagnostics, demonstrate that an understanding of truncation-error correlations between different densities and different observables is crucial for reliable UQ. The methods developed here are publicly available as annotated Jupyter notebooks.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗