Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “error quantification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Projection-Based Model Reduction for Coupled Conduction—Enclosure Radiation Systems

We report a projection-based reduced order model (pROM) methodology has been developed for transient heat transfer problems involving coupled conduction and enclosure radiation. The approach was demonstrated on two test problems of varying complexity. The reduced order models demonstrated substantial speedups (up to 185×) relative to the full order model with good accuracy (less than 3% L ∞ error). An attractive feature of pROMs is that there is a natural error indicator for the ROM solution: the final residual norm at each time-step of the converged ROM solution. Using example test cases, we discuss how to interpret this error indicator to assess the accuracy of the ROM solution. The approach shows promise for many-query applications, such as uncertainty quantification and optimization. The reduced computational cost of the ROM relative to the full-order model (FOM) can enable the analysis of larger and more complex systems as well as the exploration of larger parameter spaces.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A bootstrapping approach to social media quantification

Abstract This work considers the use of classifiers in a downstream aggregation task estimating class proportions, such as estimating the percentage of reviews for a movie with positive sentiment. We derive the bias and variance of the class proportion estimator when taking classification error into account to determine how to best trade off different error types when tuning a classifier for these tasks. Additionally, we propose a method for constructing confidence intervals that correctly adjusts for classification error when estimating these statistics. We conduct experiments on four document classification tasks comparing our methods to prior approaches across classifier thresholds, sample sizes, and label distributions. Prior approaches have focused on providing the most accurate point estimate while this work focuses on the creation of correct confidence intervals that appropriately account for classifier error. Compared to the prior approaches, our methods provide lower error and more accurate confidence intervals.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Assessment of Flow-Enhanced Electrochemical Sensor Testing and Deployments for MSRs

This report serves as the deliverable for Milestone M3RS-23AN0401061 that is part of Work Package RS-23AN040106 (Flow Enhanced Sensors for MSRs – ANL). The goal of this milestone was to determine performance of the flow enhanced electrochemical sensor (FEES) and modular flow instrumentation testbed (MFIT) in safeguards relevant scenarios. Flow enhanced electrochemical sensors are a type of electroanalytical sensor that has been developed at Argonne National Laboratory to be installed directly into MSR flow conduits to make measurements of the salt composition. These sensors represent a significant improvement in capabilities compared to earlier electroanalytical sensors that instead can only be operated in quiescent conditions. Previous work has focused on testing of the FEES in flowing conditions provided by the MFIT to assess the accuracy and precision of the sensor measurements. To further improve this capability, in FY23 we undertook a campaign of safeguards relevant scenarios in molten salt containing a range of uranium chloride concentrations (0 to 3 wt%). All the testing carried out in FY23 was aided by a control system designed to automatically actuate flow conditions and collect data. This new automation system is estimated to have increased experimental throughput by a factor of four and enabled testing in a variety of complex conditions. The advancements in throughput and repeatability led to improved quantification of actinide concentrations using the in-flow sensors, with a reduction of the mean absolute relative error from 5.6% in FY22 to 3.1% in FY23. In addition to safeguards scenarios run in the MFIT, FY23 work included deployment of a FEES at a partner institution where it will be tested in a flowing salt loop. The FEES was successfully integrated into that loop and is being tested prior to loop startup. In FY23, work also continued on the smaller flow system that we have named the mini-MFIT. This smaller system is capable of rapid prototyping of new sensor designs prior to installation in the larger MFIT radiological flow system. Work was carried out to test this new system in non-radiological molten salts in a separate glovebox. This work is helping us to enhance the accuracy of our salt monitoring capabilities through the integration of multiple types of sensors. The high degree of accuracy required by 10 CFR 74 represents a significant challenge, and further design evolution and integration of the sensors into multimodal sensing frameworks will be needed to further push the measurement accuracy to the needed level.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Sparsifying priors for Bayesian uncertainty quantification in model discovery

We propose a probabilistic model discovery method for identifying ordinary differential equations governing the dynamics of observed multivariate data. Our method is based on the sparse identification of nonlinear dynamics (SINDy) framework, where models are expressed as sparse linear combinations of pre-specified candidate functions. Promoting parsimony through sparsity leads to interpretable models that generalize to unknown data. Instead of targeting point estimates of the SINDy coefficients, we estimate these coefficients via sparse Bayesian inference. The resulting method, uncertainty quantification SINDy (UQ-SINDy), quantifies not only the uncertainty in the values of the SINDy coefficients due to observation errors and limited data, but also the probability of inclusion of each candidate function in the linear combination. UQ-SINDy promotes robustness against observation noise and limited data, interpretability (in terms of model selection and inclusion probabilities) and generalization capacity for out-of-sample forecast. Sparse inference for UQ-SINDy employs Markov chain Monte Carlo, and we explore two sparsifying priors: the spike and slab prior, and the regularized horseshoe prior. UQ-SINDy is shown to discover accurate models in the presence of noise and with orders-of-magnitude less data than current model discovery methods, thus providing a transformative method for real-world applications which have limited data.

97 MATHEMATICS AND COMPUTING↗

Development of advanced machine learning models for analysis of plutonium surrogate optical emission spectra

This work investigates and applies machine learning paradigms seldom seen in analytical spectroscopy for quantification of gallium in cerium matrices via processing of laser-plasma spectra. Ensemble regressions, support vector machine regressions, Gaussian kernel regressions, and artificial neural network techniques are trained and tested on cerium-gallium pellet spectra. A thorough hyperparameter optimization experiment is conducted initially to determine the best design features for each model. The optimized models are evaluated for sensitivity and precision using the limit of detection (LoD) and root mean-squared error of prediction (RMSEP) metrics, respectively. Gaussian kernel regression yields the superlative predictive model with an RMSEP of 0.33% and an LoD of 0.015% for quantification of Ga in a Ce matrix. This study concludes that these machine learning methods could yield robust prediction models for rapid quality control analysis of plutonium alloys.

Rao, Ashwin P. (ORCID:0000000319312568)↗

Deeply uncertain: comparing methods of uncertainty quantification in deep learning algorithms

We present a comparison of methods for uncertainty quantification (UQ) in deep learning algorithms in the context of a simple physical system. Three of the most common uncertainty quantification methods - Bayesian Neural Networks (BNN), Concrete Dropout (CD), and Deep Ensembles (DE) - are compared to the standard analytic error propagation. We discuss this comparison in terms endemic to both machine learning ("epistemic" and "aleatoric") and the physical sciences ("statistical" and "systematic"). The comparisons are presented in terms of simulated experimental measurements of a single pendulum - a prototypical physical system for studying measurement and analysis techniques. Our results highlight some pitfalls that may occur when using these UQ methods. For example, when the variation of noise in the training set is small, all methods predicted the same relative uncertainty independently of the inputs. This issue is particularly hard to avoid in BNN. On the other hand, when the test set contains samples far from the training distribution, we found that no methods sufficiently increased the uncertainties associated to their predictions. This problem was particularly clear for CD. In light of these results, we make some recommendations for usage and interpretation of UQ methods.

59 BASIC BIOLOGICAL SCIENCES↗

Robust Solution Verification Experiments on Nonuniform Meshes

The activities of verification, validation, and uncertainty quantification (VVUQ) provide a comprehensive means to assess the credibility of computational models. Within VVUQ, solution verification assesses numerical errors and evaluates whether the simulation is sufficiently accurate for its intended applications. As computational modeling gains traction in the development of complex, high-consequence systems, the need for robust solution verification intensifies, particularly because experimental data for these systems are often limited. This work examines improvements in the robustness of Richardson extrapolation (RE), a method commonly used in solution verification to study the discretization error of computational models using a power law. Nonuniform mesh refinement is discussed alongside other pollutants that affect the robustness of the power law model. Maximum likelihood estimation (MLE) is proposed as a robust strategy to address the uncertainty generated by nonuniform mesh refinement. An exploratory computational fluid dynamics (CFD) study of a 2D planar Poiseuille flow is conducted to determine if nonuniform mesh noise can be modeled with this MLE approach for more robust RE.

Weinmeister, Justin [ORNL] (ORCID:0000000160090237↗

A proposed high-intensity radiometer calibration method using concentrated solar radiation

Accurate calibration of irradiance measurement devices, or radiometers, is essential for ensuring the reliability of measurements in high heat applications such as concentrating solar power (CSP), aerospace, defense, and pulsed power systems. Despite the critical need, existing calibration standards and service providers are limited to irradiance levels below 100 kW/m 2 and specific radiation sources, which is insufficient for many applications. For instance, CSP technologies, particularly those under the Department of Energy’s Solar Energy Technologies Office (SETO) Gen 3 program, require accurate measurements of broadband irradiance at levels exceeding 2000 kW/m 2 . In even more extreme scenarios, such as re-entry vehicles, heat levels can surpass 10000 kW/m 2 . Current ISO standards, specifically ISO 14934–2 and ISO 14934–3, are constrained to lower irradiance levels and dependent on black body heat sources, limiting their applicability for high-intensity broadband irradiance measurements, particularly in concentrated solar applications. Here, to address this shortfall, the National Solar Thermal Test Facility (NSTTF) at Sandia National Laboratories (SNL) proposes a calibration method and facility capable of characterizing radiometers up to 2750 kW/m 2 using concentrated solar irradiance. Calibrating with concentrated sunlight is important for solar applications as it aligns the calibration process with the solar spectrum. This alignment is crucial for minimizing systematic errors and avoiding the need for additional corrections that may arise when radiometers designed for solar applications are calibrated using black-body or electrical sources. This paper presents the present day NSTTF characterization facility and procedure, detailing the proposed calibration method and uncertainty quantification. The presented method builds upon 1980′s NSTTF methodology and involves both theoretical and empirical methods to establish a robust relationship between gauge voltage output and irradiance intensity, quantifying both measurement and fitting errors. By addressing the limitations of existing standards and extending the characterization range, this work provides an advancement in the field of high-intensity irradiance measurement and instrumentation characterization.

Gardon gauge↗

Emulating climate extreme indices

We use simple pattern scaling and time-shift to emulate changes in a set of climate extreme indices under future scenarios, and evaluate the emulators' accuracy. We propose a metric for the error in emulation in the context of initial condition ensembles, to specifically characterize the role of internal variability in the emulation performance. Our metric separates systematic emulation errors from unavoidable discrepancies between emulated and target values due to internal variability. We compute the metricis at grid-point scale, and we show geographically resolved results, or aggregate them at global scale. We demonstrate the use of our error metric in the emulation of a suite of temperature and precipitation extreme indices. We test and compare simple pattern scaling and time-shift using a range of trajectories spanning targets inspired by the Paris agreement -- warming to 1.5C and 2.0C from the pre-industrial baseline -- and two of the longer-established trajectories, RCP4.5 and RCP8.5. With this suite of scenarios we can test the effects on the performance of the size of the temperature gap between emulation origin and target. We find that for most indices emulation the dominant source of discrepancy is internal variability. For at least one index, however, counting exceedances of a high temperature threshold, significant portions of the globally aggregated discrepancy and its regional pattern originate from the systematic emulation error. This error exceeds internal variability of both the target and the emulated quantities in large coherent regions at low latitudes, and the explanation can be found in the differential behavior of temperature distributions across latitudes. The metric also highlights a fundamental difference in the two methods related to the simulation of internal variability, which is dampened significantly by simple pattern scaling. This aspect is of consequence when using these methods for specific applications, where preserving variability for uncertainty quantification is deemed important. With this study we offer our metric as a diagnostic tool, facilitating the formulation of scientific hypotheses on the reasons for the error. In the meantime, we show that for many impact relevant indices by now traditional emulation techniques can be accurate within the variations unavoidably introduced by internal variability, establishing the fundamental condition for using their emulation in impact modeling.

54 ENVIRONMENTAL SCIENCES↗

Uncertainty-Aware and Explainable Human Error Detection in the Operation of Nuclear Power Plants

The timely and accurate identification of incidents, such as human factor error, is important to restore nuclear power plants (NPPs) to a stable state. However, the identification of abnormal operating conditions is difficult because of the existence of multiple scenarios. In addition, to implement mitigation actions rapidly after an incident occurs, operators must accurately identify an incident by monitoring the trends of many variables. The mental burden posed by this can increase human error and cause failure in identifying incidents. Failure to identify incidents directly results in erroneous mitigation measures, which are detrimental to NPPs. In this study, we leverage uncertainty-aware models to identify such errors and thereby increase the chances of mitigating them. We use the data collected from a physical test bed. The goal is to identify both certain and accurate models. For this, the two main aspects of focus in this study are explainable artificial intelligence (XAI) and uncertainty quantification (UQ). While XAI elucidates the decision pathway, UQ evaluates decision reliability. Their integration paints a comprehensive picture, signifying that understanding decisions and their confidence should be interlinked. Thus, in this study we leverage UQ measures (e.g. entropy and mutual information) along with Shapley additive explanations to gain insights into the features contributing to both accuracy and uncertainty in error identification. Furthermore, our results show that uncertainty-aware models combined with XAI tools can explain the artificial intelligence–prescribed decisions, with the potential of better explaining errors for the operators.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Assessment of Clonal Expansion Using CarcSeq Measurement of Lung Cancer Driver Mutations and Correlation With Mouse Strain- and Sex-Related Incidence of Spontaneous Lung Neoplasia

Quantification of variation in levels of spontaneously occurring cancer driver mutations (CDMs) was developed to assess clonal expansion and predict future risk of neoplasm development. Specifically, an error-corrected next-generation sequencing method, CarcSeq, and a mouse CarcSeq panel (analogous to human and rat panels) were developed and used to quantify low-frequency mutations in a panel of amplicons enriched in hotspot CDMs. Mutations in a subset of panel amplicons, Braf, Egfr, Kras, Stk11, and Tp53, were related to incidence of lung neoplasms at 2 years. This was achieved by correlating median absolute deviation (MAD) from the overall median mutant fraction (MF) measured in the lung DNA of 16-week-old male and female, B6C3F1 and CD-1 mice (10 mice/sex/strain) with percentages of spontaneous alveolar/bronchioloalveolar adenomas and carcinomas reported in bioassay control groups. A total of 1586 mouse lung mutants with MFs > x 10-4 were recovered. The ratio of nonsynonymous to synonymous mutations was used to assess the proportion of recovered mutations conferring a positive selective advantage. The greatest ratio was observed in what is considered the most lung tumor-sensitive model examined, male B6C3F1 mice. Of the recurrent, nonsynonymous mouse mutations recovered, 55.5% have been reported in human tumors, with many located in or around the mouse equivalent of human cancer hotspot codons. MAD for the same subset of amplicons measured in normal human lung DNA samples showed a correlation of moderate strength and borderline significance with age (a cancer risk factor), as well as age-related cumulative lung cancer risk, suggesting MAD may inform species extrapolation.

Toxicology↗

Meta-uncertainty for particle image velocimetry

Uncertainty quantification for Particle Image Velocimetry (PIV) is critical for comparing experimentally measured flow fields with Computational Fluid Dynamics (CFD) results, and model design and validation. However, PIV features a complex measurement chain with coupled, non-linear error sources, and quantifying the uncertainty is challenging. Multiple assessments show that none of the current methods can reliably measure the actual uncertainty across a wide range of experiments, and estimates can vary. Because the current methods differ in assumptions regarding the measurement process and calculation procedures, it is not clear which method is best to use for an experiment where the error distribution is unknown. To address this issue, we propose a method to estimate an uncertainty method's sensitivity and reliability, termed the Meta-Uncertainty. The novel approach is automated, local, and instantaneous, and based on perturbation of the recorded particle images. We developed an image perturbation scheme based on adding random unmatched particles to the interrogation window pair considering the signal-to-noise (SNR) of the correlation plane. Each uncertainty scheme's response to several trials of random particle addition is used to estimate a reliability metric, defined as the rate of change of the inter-quartile range (IQR) of the uncertainties with increasing levels of particle addition. We also propose applying the meta-uncertainty as a weighting metric to combine uncertainty estimates from individual schemes, based on ideas from the consensus forecasting literature. We use planar and stereo PIV measurements across a range of canonical flows to assess the performance of the uncertainty schemes. Further, a novel method is introduced to assess an uncertainty scheme's performance based on a quantile comparison of the error and uncertainty distributions, generalizing the current method of comparing the RMS of the two distributions. Here, the results show that the combined uncertainty method outperforms the individual methods, and this work establishes the meta-uncertainty as a useful reliability assessment tool for PIV uncertainty quantification.

47 OTHER INSTRUMENTATION↗

Bottom-up and Top-Down Uncertainty Quantification for Measurements

Several recent papers address improved uncertainty quantification (UQ) for measurements used in nuclear safeguards. This paper reviews progress and presents new results for bottom-up (first principles) and top-down (empirical) UQ for safeguards, where the main quantitative measure of uncertainty is the total measurement error standard deviation (SD), which includes both random and systematic error components. The five main UQ topics addressed here include: (1) impact of making data-driven choices in SD estimation; (2) use of approximate Bayesian computation (ABC) for both bottom-up and top-down UQ; (3) computational calibration; (4) revisions to the guide to the expression of uncertainty in measurement (GUM), and (5) critique of a recently-suggested “Unified Theory of Measurement Errors and Uncertainties.”

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Orphaned oil and gas well methane emission rates quantified using Gaussian plume inversions of ambient observations

Abstract. Annually, ∼ 3.6 million abandoned oil and gas wells in the US emit a combined ∼ 2.6 Tg methane (CH4), adversely affecting climate and regional air quality. However, these estimates depend on emission factors derived from measuring subpopulations of wells that vary by orders of magnitude due to very limited field sampling and poorly characterized distributions. Currently, US protocols to remediate orphaned wells lacks standardized quantification methods needed to both prioritize plugging and account for emission reductions. Therefore, sensitive, reliable, affordable, and scalable CH4 flux quantification methods are needed. We report the use of a simple Gaussian plume method where the dispersion parameters are constrained by in situ ground measurements of CH4 concentration at four locations 7.5–49 m downwind of the orphan well as well as local winds to estimate the leak rate from an orphan well. We derive a flux of 10.53 ± 1.16 kg CH4 h−1 during a venting procedure in April 2023 that agrees with the directly measured volumetric flow rate of 9.00 ± 0.25 kg CH4 h−1. This is 71 % greater than the 5.3 kg CH4 h−1 flux measured 7 months prior. Additionally, we discovered a secondary leak through the surface casing inferred as 0.43–0.67 kg CH4 h−1 both by our ground Gaussian analysis and by transecting the plume with an uncrewed aerial system (UAS). We show that in situ determination of the dispersion parameters used in our Gaussian inversions allows us to measure methane emissions to 15 % accuracy, significantly reducing errors when compared to the standard practice of assuming stability class. Our results help develop simpler methods and protocols for robust orphan well emission quantification that can be used for reporting.

Follansbee, Emily↗

Pyrrole‐Imine Macrocycle: Self‐Organizing Cross‐Reactive Anion Receptor and Sensor

Self-organizing macrocyclic receptor-sensors for phosphorus oxyanions, phosphates, and phosphonates comprising imine moieties were prepared by condensation of dipyrrolylmethane dicarbaldehyde with diethylene triamine. The incorporation of flexible ethylene moieties endows the macrocycle with unprecedented flexibility and ability to accommodate numerous phosphorus oxyanions from orthophosphate to large anions such as ATP or phosphonate glyphosate. The anion binding was elucidated by NMR titrations, low-temperature NMR, and NOESY NMR. The incorporation of dansyl fluorophore enables sensing of anions using the fluorescence signal, whereas the changes in fluorescence intensity, width of the fluorescence band, and position of the maxima are analyte-specific and useful in recognition and identification of eleven different P-oxyanions in water. The affinity (K assoc ) for Na + salts was H 2 PO 4 − ≈ Methylphosphonate > H 2 P 2 O 7 2− > Phenylphosphonate- > Glyphosate 2− > AMP 2− > ADP 2− > ATP 2− . Interestingly, phosphonates, including methylphosphonate and glyphosate anions, were also found to display a strong affinity (K assoc ∼10 6 M −1 ) while halides, nitrate, carbonates, or hydrogen sulfate did not show a significant affinity. The determined fluorescence spectral parameters were used to classify the 12 analytes (11 anions and water) using Linear Discriminant Analysis (LDA). Quantification was performed using LDA and Support Vector Machine (SVM), and the phosphonate concentrations in unknown samples were determined with an error of 3.5% or lower.

anions↗

Stochastic Optimization and Uncertainty Quantification of Natrium-based Nuclear-Renewable Energy Systems for Flexible Power Applications in Deregulated Markets

Rapid integration of variable renewable energy sources (VRES) has made modeling and stochastic optimization of hybrid energy systems crucial for studying their long-term performance and viability. However, most studies have focused on just historical data, which may be unreliable for capturing short-term fluctuations, rare events, and long-term patterns of energy demand, price, and the variability of renewable energy sources. For this study, optimal synthetic time series models were developed using Wasserstein distance. The models were validated by comparing the key statistical measures against those of the historical data. They were then used to optimize the integrated Natrium-style advanced energy systems and their long-term (30 years) economics. The stochastic model performs bi-level optimization to find the optimal sizes for the balance of plant and thermal energy storage, while also optimizing energy dispatch to achieve the maximum net present value. In studies of two deregulated markets (California ISO and the Electric Reliability Council of Texas), the integrated Natrium-style system performed better in CAISO than in ERCOT, given higher and more consistent electricity prices during peak-demand periods. The potentially enlarged cost associated with the variable operation and maintenance of the TES system also plays a significant role in driving the system sizing, thus its impacts on the system are investigated in detail through comparison against a baseline case. The study also finds that the bi-level optimization results based on stochastic gradient descent closely match the grid search results. The uncertainty quantification of the stochastic signals provides further NPV-related insights and probability distributions for the case studies. The normal standard error of the mean of NPV for the case with and without TES VOM for CAISO were found to be 7.73M (plus-minus sign) 1.09M USD and 104.99M (plus-minus sign) 1.25M USD, respectively based on a 95% confidence. Given the relatively small NPV variance based on 150 samples, the analysis affords the most robust possible prediction of the techno-economic performance of the integrated Natrium-style energy systems.

25 ENERGY STORAGE↗

Generalized Bayesian MARS: Tools for Stochastic Computer Model Emulation

The multivariate adaptive regression spline (MARS) approach of Friedman and its Bayesian counterpart are effective approaches for the emulation of computer models. The traditional assumption of Gaussian errors limits the usefulness of MARS, and many popular alternatives, when dealing with stochastic computer models. Here, we propose a generalized Bayesian MARS (GBMARS) framework which admits the broad class of generalized hyperbolic distributions as the induced likelihood function. This allows us to develop tools for the emulation of stochastic simulators which are parsimonious, scalable, and interpretable and require minimal tuning, while providing powerful predictive and uncertainty quantification capabilities. GBMARS is capable of robust regression with t distributions, quantile regression with asymmetric Laplace distributions, and a general form of “Normal-Wald” regression in which the shape of the error distribution and the structure of the mean function are learned simultaneously. We demonstrate the effectiveness of GBMARS on various stochastic computer models, and we show that it compares favorably to several popular alternatives.

97 MATHEMATICS AND COMPUTING↗

Weighted greedy-optimal design of computer experiments for kernel-based and Gaussian process model emulation and calibration

This article is concerned with the approximation of high-dimensional functions by kernel-based methods. Motivated by uncertainty quantification, which often necessitates the construction of approximations that are accurate with respect to a probability density function of random variables, we aim at minimizing the approximation error with respect to a weighted $L^p$-norm. We present a greedy procedure for designing computer experiments based upon a weighted modification of the pivoted Cholesky factorization. The method successively generates nested samples with the goal of minimizing error in regions of high probability. Numerical experiments validate that this new importance sampling strategy is superior to other sampling approaches, especially when used with non-product probability density functions. We also show how to use the proposed algorithm to efficiently generate surrogates for inferring unknown model parameters from data.

97 MATHEMATICS AND COMPUTING↗