SEARCH · Engineering Papers

Results for “error quantification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Development of the uncertainty quantification toolkit's python interface and surrogate construction tutorial

The uncertainty quantification toolkit (UQTk) is a collection of c++ libraries that assess the confidence of numerical models. Surrogate approximations, often polynomial chaos expansions (PCEs), lessen the computational cost of these assessments. I developed a Python interface in UQTk for regression and Bayesian compressive sensing to add to the existing Galerkin projection method. These methods receive an object containing the polynomial basis information and NumPy arrays of sample points, call c++ methods, and return the PCE coefficients in a NumPy array. To demonstrate these methods, I wrote a tutorial in which I use them to construct surrogates for Genz functions and calculate the resulting error.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Connecting ground-state properties of 6 Li to each other and to scattering data

We examine the relationship between the asymptotic normalization coefficient (ANC) of 6 Li and other low-energy observables in the α–deuteron system. Our analysis uses a set of calculations carried out within the ab initio no core shell model with continuum (NCSMC) using a variety of inter-nucleon interactions and basis sizes, and yielding 6 Li deuteron separation energies between 1.3 and 1.8 MeV (Hebborn et al 2022 Phys. Rev. Lett. 129 042503). These NCSMC calculations show that the square of the ANC is strongly correlated with the separation energy over this range. In this work, we investigate the origin of this correlation using the phenomenological R-matrix, a single-channel potential and a perturbative approach. We show that this correlation occurs because the depth of the α–deuteron central potential changes by only a small relative amount as the separation energy varies. We then investigate if the ANC can be accurately extracted from α–deuteron phase shifts in an ideal case in which low-energy data are available and there are no experimental errors. We find that both R-matrix and Coulomb-modified effective-range theory (CM-ERE) yield extracted ANCs close to, although not exactly equal to, the NCSMC value, provided the extrapolation is constrained by the known position of the bound-state pole and at least three terms are included in the fit function. The R-matrix approach converges faster than the CM-ERE as the number of parameters increases and is also more robust against the inclusion of low-energy and high-energy phase shift data. Finally, our study also shows that a naive quantification of uncertainties by comparing different truncations used in both theories is not accurate, and suggests the accuracy of ANCs extracted from phase shift data needs further investigation.

R-matrix↗

DOE OSTI2024

Ensemble Kalman inversion of induced polarization data

SUMMARY This paper explores the applicability of ensemble Kalman inversion (EKI) with level-set parametrization for solving geophysical inverse problems. In particular, we focus on its extension to induced polarization (IP) data with uncertainty quantification. IP data may provide rich information on characteristics of geological materials due to its sensitivity to characteristics of the pore–grain interface. In many IP studies, different geological units are juxtaposed and the goal is to delineate these units and obtain estimates of unit properties with uncertainty bounds. Conventional inversion of IP data does not resolve well sharp interfaces and tends to reduce and smooth resistivity variations, while not readily providing uncertainty estimates. Recently, it has been shown for DC resistivity that EKI is an efficient solver for inverse problems which provides uncertainty quantification, and its combination with level set parametrization can delineate arbitrary interfaces well. In this contribution, we demonstrate the extension of EKI to IP data using a sequential approach, where the mean field obtained from DC resistivity inversion is used as input for a separate phase angle inversion. We illustrate our workflow using a series of synthetic and field examples. Variations with uncertainty bounds in both DC resistivity and phase angles are recovered by EKI, which provides useful information for hydrogeological site characterization. Although phase angles are less well-resolved than DC resistivity, partly due to their smaller range and higher percentage data errors, it complements DC resistivity for site characterization. Overall, EKI with level set parametrization provides a practical approach forward for efficient hydrogeophysical imaging under uncertainty.

Geochemistry & Geophysics↗

DOE OSTI2020

Assessment of the frequency and nature of erroneous x-ray photoelectron spectroscopy analyses in the scientific literature

Here, this study was undertaken to understand the extent and nature of problems in x-ray photoelectron spectroscopy (XPS) data reported in the literature. It first presents an assessment of the XPS data in three high-quality journals over a six-month period. This analysis of 409 publications showing XPS spectra provides insight into how XPS is being used, identifies the common mistakes or errors in XPS analysis, and reveals which elements are most commonly analyzed. More than 65% of the 409 papers showed fitting of XP spectra. An ad hoc group (herein identified as “the committee”) of experienced XPS analysts reviewed these spectra and found that peak fitting was a common source of significant errors. The papers were ranked based on the perceived seriousness of the errors, which ranged from minor to major. Major errors, which, in the opinion of the ad hoc committee, can render the interpretation of the data meaningless, occurred when fitting protocols ignored underlying physics and chemistry or contained major errors in the analysis. Consistent with other materials analysis data, ca. 30% of the XPS data or analysis was identified as having major errors. Out of the publications with fitted spectra, ca. 40% had major errors. The most common elements analyzed by XPS in the papers sampled and researched at an online database, include carbon, oxygen, nitrogen, sulfur, and titanium. A scrutiny of the papers showing carbon and oxygen XPS spectra revealed the classes of materials being studied and the extent of problems in these analyses. As might be expected, C 1s and O 1s analyses are most often performed on sp2-type materials and inorganic oxides, respectively. These findings have helped focus a series of XPS guides and tutorials that deal with common analysis issues. The extent of problematic data is larger than the authors had expected. Quantification of the problem, examination of some of the common problem areas, and the development of targeted guides and tutorials may provide both the motivation and resources that enable the community to improve the overall quality and reliability of XPS analysis reported in the literature.

Major, George H.↗

DOE OSTI2024

A robust approach to Gaussian process implementation

Abstract. Gaussian process (GP) regression is a flexible modeling technique used to predict outputs and to capture uncertainty in the predictions. However, the GP regression process becomes computationally intensive when the training spatial dataset has a large number of observations. To address this challenge, we introduce a scalable GP algorithm, termed MuyGPs, which incorporates nearest-neighbor and leave-one-out cross-validation during training. This approach enables the evaluation of large spatial datasets with state-of-the-art accuracy and speed in certain spatial problems. Despite these advantages, conventional quadratic loss functions used in the MuyGPs optimization, such as root mean squared error (RMSE), are highly influenced by outliers. We explore the behavior of MuyGPs in cases involving outlying observations and, subsequently, develop a robust approach to handle and mitigate their impact. Specifically, we introduce a novel leave-one-out loss function based on the pseudo-Huber function (LOOPH) that effectively accounts for outliers in large spatial datasets within the MuyGPs framework. Our simulation study shows that the LOOPH loss method maintains accuracy despite outlying observations, establishing MuyGPs as a powerful tool for mitigating unusual observation impacts in the large data regime. In the analysis of US ozone data, MuyGPs provides accurate predictions and uncertainty quantification, demonstrating its utility in managing data anomalies. Through these efforts, we advance the understanding of GP regression in spatial contexts.

Mukangango, Juliette↗

DOE OSTI2022

PI3NN: Out-of-distribution-aware Prediction Intervals from Three Neural Networks

We propose a novel prediction interval (PI) method for uncertainty quantification, which addresses three major issues with the state-of-the-art PI methods. First, existing PI methods require retraining of neural networks (NNs) for every given confidence level and suffer from the crossing issue in calculating multiple PIs. Second, they usually rely on customized loss functions with extra sensitive hyperparameters for which fine tuning is required to achieve a well-calibrated PI. Third, they usually underestimate uncertainties of out-of-distribution (OOD) samples leading to over-confident PIs. Our PI3NN method calculates PIs from linear combinations of three NNs, each of which is independently trained using the standard mean squared error loss. The coefficients of the linear combinations are computed using root-finding algorithms to ensure tight PIs for a given confidence level. We theoretically prove that PI3NN can calculate PIs for a series of confidence levels without retraining NNs and it completely avoids the crossing issue. Additionally, PI3NN does not introduce any unusual hyperparameters resulting in a stable performance. Furthermore, we address OOD identification challenge by introducing an initialization scheme which provides reasonably larger PIs of the OOD samples than those of the in-distribution samples. Benchmark and real-world experiments show that our method outperforms several state-of-the-art approaches with respect to predictive uncertainty quality, robustness, and OOD samples identification.

Liu, Siyan↗

DOE OSTI2020

Analysis of Random Forest Modeling Strategies for Multi-Step Wind Speed Forecasting

Although the random forest (RF) model is a powerful machine learning tool that has been utilized in many wind speed/power forecasting studies, there has been no consensus on optimal RF modeling strategies. This study investigates three basic questions which aim to assist in the discernment and quantification of the effects of individual model properties, namely: (1) using a standalone RF model versus using RF as a correction mechanism for the persistence approach, (2) utilizing a recursive versus direct multi-step forecasting strategy, and (3) training data availability on model forecasting accuracy from one to six hours ahead. These questions are investigated utilizing data from the FINO1 offshore platform and Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) C1 site, and testing results are compared to the persistence method. At FINO1, due to the presence of multiple wind farms and high inter-annual variability, RF is more effective as an error-correction mechanism for the persistence approach. The direct forecasting strategy is seen to slightly outperform the recursive strategy, specifically for forecasts three or more steps ahead. Finally, increased data availability (up to ~8 equivalent years of hourly training data) appears to continually improve forecasting accuracy, although changing environmental flow patterns have the potential to negate such improvement. We hope that the findings of this study will assist future researchers and industry professionals to construct accurate, reliable RF models for wind speed forecasting.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Advanced Graphite Creep Uncertainty Analysis

Radiation damage estimation is an important component of the post irradiation analysis of the Advanced Graphite Creep (AGC) experiment. It depends primarily on the fast fluence, which is determined using well established methods of spectral adjustment. These are based on best estimates from models such as Monte Carlo N-Particle (MCNP), input cross-sections, and measured activities from flux wires in the experiment. Each of these parameters can propagate uncertainties which will affect the uncertainty in the calculated dose levels for AGC, or any experiment irradiated within a reactor. While the methods of propagating uncertainty are well-established, the final uncertainty estimates they provide are only as good as the estimates of uncertainty in the inputs on which they are based. The purpose of this work is to outline some deficiencies in the ways these input uncertainties are presently estimated, and to outline a methodology by which they can be improved. The fast fluence and radiation damage received by graphite specimens irradiated in the Advanced Graphite Creep (AGC) experiments is presently estimated using spectral adjustment methods that are based on both flux wire activity measurements, and MCNP model predictions. This work describes an ongoing effort to quantify and propagate uncertainties in inputs to the spectral adjustment process, and thereby quantify the resultant error in radiation damage (dpa) estimates. The effort is multi-faceted, and we consider the impacts of both the set of flux wires selected, and the counting process. An expanded set of flux wires is identified that provides a more comprehensive data set on the fast spectrum. To address the counting process itself, a series of round-robin measurements in several reactor metrology laboratories across the Department of Energy (DOE) complex and nuclear industry are being undertaken to refine the American Society for Testing and Materials (ASTM) standards for flux wire measurements. To address the contribution of uncertainty in the MCNP model predictions, an uncertainty quantification (UQ) tool has been developed that statistically samples the model input parameters, runs a series of cases, and assimilates the results to provide an overall uncertainty. The impact of the MCNP UQ tool results is demonstrated by re-analyzing previous AGC flux wire and irradiation data. While the expanded flux wire set obviously cannot be added to these experiments retroactively, plans for future graphite irradiations are outlined.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2024

Systematic exploration of the thermochemistry for a set of peroxy hydroperoxy-alkyl radicals

Here, the thermochemistry of peroxy hydroperoxy-alkyl ($\overline{O}$OQOOH) radicals has a significant influence on the reactivity of fuels and on the formation of highly oxygenated molecules (HOMs) in the atmosphere. Theoretical characterization of these radicals can be arduous due to their molecular size and complex fundamental interactions, such as hydrogen-bonding and torsional anharmonicity, and difficult to validate in the absence of any direct experimental thermochemical data. In this work, we systematically explore the thermochemistry of a set of $\overline{O}$OQOOH radicals with five increasingly affordable approaches with considerations for these interactions. In doing so, we present a novel conformer selection approach suited to predict properties at combustion temperatures. As a corollary, we also highlight the shortcomings in the standard choice of the ground conformer as the reference. The set of molecules is comprised of 149 C 2 -C 8 $\overline{O}$OQOOH radicals, selected to encompass a wide variety of branching and substitution patterns. Comparisons amongst the approaches help quantify the errors arising from various simplifying assumptions. For the largest of these radicals, and with the most affordable approach, final predictions of Gibbs energies are assigned a 2 σ uncertainty of 4 kcal mol -1 in the negative temperature coefficient region.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

Uncertainty Quantification of Metal Additive Manufacturing Processing Conditions Through the use of Exascale Computing

Metal additive manufacturing (AM) is a disruptive manufacturing technology that opens the design space for parts outside those possible from traditional manufacturing methods. In order to accelerate industry and R&D needs to certify AM parts, the Exascale Additive Manufacturing project (ExaAM) has developed a suite of exascale-ready computational tools to model the process-to-structure-to-properties (PSP) relationship for additively manufactured metal components. One such tool is an uncertainty quantification (UQ) pipeline to quantify the effect that uncertainty in processing conditions has on local mechanical responses. We present an overview of this pipeline and its required simulation and workflow codes. Using the Oak Ridge National Laboratory’s (ORNL) exascale computer, Frontier, we utilize this pipeline to cross multiple length and time scales to predict the local mechanical response of a location within a complex AM bridge part, AMB2018-01 produced by the National Institute of Standards and Technology (NIST) as part of their 2018 AM-Bench test series. Our results are then compared to experimental mechanical tests of parts from the NIST build to quantify the error in the ExaAM UQ workflow.

Carson, Robert↗

DOE OSTI2025

Development of a River Dynamical Core for E3SM to simulate compound flooding on Exascale-class heterogeneous supercomputers

Flooding events pose significant risk to human life, property, and infrastructure. Physically-consistent quantification of altered flood risks in global models requires hyper-resolution (~1 km) or fine flood simulations using two-dimensional (2D) physics schemes, both of which are unavailable in the current generation Earth System Models. Here, in this work, we have developed the River Dynamical Core (RDycore), which is an open-source, 2D shallow water equation (SWE) library for the U.S. Department of Energy's Energy Exascale Earth System Model (E3SM). RDycore uses PETSc and libCEED libraries that allows it to run efficiently on CPUs and GPUs, as well as select a time-integration algorithm at runtime without requiring any code modifications. RDycore achieves spatial error convergence rates for problems with analytical and manufactured solutions similar to those reported previously in the literature, or consistent with the implemented first-order spatial discretization scheme. RDycore's accuracy in predicting flooding for a well-studied dam break problem is comparable to existing SWE models. For a problem with 471 million grid cells, RDycore achieves a speedup of 6.6x and 7.6x on GPUs compared to CPUs when using 320 compute nodes on DOE's Perlmutter and Frontier supercomputers, respectively. The one-way coupling of the RDycore library within E3SM is demonstrated by performing multiple 5-day flooding simulations during Hurricane Harvey driven by five precipitation datasets. The E3SM--RDycore simulations at 30 m spatial resolution accurately simulate maximum water height during the hurricane when benchmarked against a previously published study and achieve a speedup of 15x (Perlmutter) and 21x (Frontier) on GPUs relative to CPUs. The work presented here is the foundational step in providing hardware and algorithmic portability framework for simulating kilometer-scale river dynamics within E3SM.

Flood Simulation↗

DOE OSTI2025

Cluster-Graph Fingerprinting: A Framework for Quantitative Analysis of Machine-Learned Interatomic Model Training and Simulation Data

Machine-learned interatomic models represent a significant advancement in simulation methods, extending the predictive ability of first-principles methods to previously inaccessible length and time scales. However, the data-driven nature of these models can lead to difficult-to-detect errors that can compromise prediction accuracy. To address this challenge, we introduce a novel fingerprinting approach based on the Chebyshev Interaction Model for Efficient Simulation (ChIMES) ML-IAM graph-based descriptor. Our strategy enables efficient and statistically rigorous analysis of system configurations used in ML-IAM training and those generated by their application, e.g., in molecular dynamics simulations. We demonstrate that these fingerprints can effectively assess novelty of a configuration relative to an existing data set and determine dissimilarity among individual configurations, which are two key tasks in workflows for active learning-based ML-IAM training, data set curation, and on-the-fly uncertainty quantification.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗

DOE OSTI2022

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images: Preprint

Bayesian model averaging↗

DOE OSTI2022

Maximum Likelihood Spectrum Decomposition for Isotope Identification and Quantification

A spectral decomposition method has been implemented to identify and quantify isotopic source terms in high-resolution gamma-ray spectroscopy in static geometry and shielding scenarios. Monte Carlo simulations were used to build the response matrix of a shielded high-purity germanium detector monitoring an effluent stream with a Marinelli configuration. The decomposition technique was applied to a series of calibration spectra taken with the detector using a multi-nuclide standard. These results are compared with decay-corrected values from the calibration certificate. For most nuclei in the standard ( 241 Am, 109 Cd, 137 Cs, and 60 Co), the deviations from the certificate values were generally no more than 6% with a few outliers as high as 10%. Furthermore, for 57 Co, the radionuclide with the lowest activity, the deviations from the standard reached as high as 25%, driven by the meager statistics in the calibration spectra. In addition, a complete treatment of error propagation for the technique is presented.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

Evaluating Probabilistic Deep Learning Methods for Uncertainty Quantification of Precipitation Bias Correction

Climate models often exhibit biases in their precipitation predictions, particularly underestimating high-intensity events and overestimating low precipitation. Deep learning approaches offer promising solutions, but their epistemic uncertainty associated with a deep learning–based bias correction method has not previously been quantified for reliable downstream climate impact studies. While methods for capturing the epistemic uncertainty in deep learning frameworks exist, there is currently no consensus on the best method. In this work, we compare three uncertainty quantification (UQ) methods—Deep Ensembles (DEns), Monte Carlo Dropout (MCD), and Flipout—by assessing the reliability of their uncertainty estimates using standard measures such as sharpness and calibration. These UQ methods are applied to an existing deep learning precipitation bias correction model known as UFNet: a coupled U-Net and fully connected neural network. The methods utilized to assess the models’ uncertainties are 1) calibration, which ensures that the expected probabilities of the model align with reality and 2) sharpness, which is a measure of the precision of the model’s probabilistic predictions. Of the three UQ methods evaluated, the DEns and MCD methods demonstrated the best-calibrated performance (expected calibration error of 0.36 and 0.35, respectively), compared to Flipout (0.58). In contrast, Flipout had the sharpest predictions and the highest metric performance in bias correcting precipitation—especially for higher-order moments such as kurtosis with a spatial correlation of 72% compared to 32% and 55% spatial correlation for DEns and MCD, respectively. Of the three UQ methods, MCD was found to be the most suitable method for UQ purposes based on its calibration, sharpness, and computational requirements.

Bayesian methods↗

DOE OSTI2020

Regulatory Decision-Making Process Using 'Weight of Evidence: An Evidence Integration Approach' by the State of Washington at the Hanford Site - 20163

Performance Assessment and Environmental Impact Statement Modeling, Human and Ecological Risk Assessments are the tools used by the Washington State Department of Ecology (Ecology) to make regulatory decisions for permitting and cleanup. These efforts all inherently have large amounts of uncertainty no matter how many alternative approaches are attempted. Professional feedback from tribal governments, stakeholders, federal and state regulators, the public and independent peer review groups are taken into account in this process of informed decision making. The process also includes full evaluation of the comments and the associated reviews made by the various peer review groups such as the Low -Level Waste Disposal Facility Federal Review Group (DOE order 435.1) and the Nuclear Regulatory Commission (NRC). The evolution of technology has produced quicker and more efficient fate and transport modeling tools with related quantification of uncertainties and their significance. Groundwater and vadose zone models have been expanded across the entire Hanford site. Many of the technical approaches identified in the Washington State Regulations are updated using current knowledge to better understand the complexities of the Hanford Site using the latest state of the art tools for risk assessment and modeling. More emphasis and time are directed on defining uncertainty and then managing the uncertainty. One added tool is to run sensitivity cases based on informed decision making. The weight of evidence approach considers all relevant information in an integrative assessment that takes into account the kinds of evidence available, the quality and quantity, the uncertainty and error associated with each type, and how they fit together. Evidence of integration involves looking at systematic reviews and evaluating their strengths and weaknesses. In conclusion, Ecology, Washington State's lead regulatory agency at the Hanford site, has had to modify its site closure approach compared to the rest of the state. A more holistic approach dealing with the Hanford site as an integrated system has evolved leading to a defensible informed approach for regulatory closure decisions. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2023

Quantification of Physical and Numerical Mixing in a Coastal Ocean Model Using Salinity Variance Budgets

Numerical mixing, the spurious mixing primarily generated by the discretization of advection, is often significant in estuarine and coastal models due to sharp, energetic fronts. In this study, we compare on- and offline estimates of numerical mixing in a submesoscale-resolving realistic simulation of the ocean state over the Texas-Louisiana continental shelf. While offline estimates of numerical mixing differ from online estimates, offline methods may be the only analysis available. This study offers insight into the differences between the on- and offline methods. We use two methods to estimate numerical mixing offline, based on salinity squared $s^2$ and volume-mean salinity variance $s^{\prime^2}$. Numerical mixing estimated from the $s^{\prime^2}$ budget is generally within 60\% of the magnitude for the online method but captures the temporal variability well. However, the $s^2$ budget compares poorly due to larger truncation errors associated with the tendency and advection terms, which can be reduced by increasing the model output frequency. We also investigate the effects of horizontal resolution on numerical mixing using a two-way nested grid. The volume-integrated numerical mixing constitutes 57\% of the bulk physical mixing -- the mixing prescribed by the turbulence closure scheme -- in the coarse model and may exceed the physical mixing by half an order of magnitude. We find that numerical mixing is reduced by 35\% on average in the nested model, less than expected based on scaling of the numerical mixing for an upwind advection scheme, likely due to new dynamical processes that emerge in the nested simulation.

54 ENVIRONMENTAL SCIENCES↗