Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “uncertainty estimation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Quantification of the importance of the modeling fidelity in the Best Estimate Plus Uncertainty analysis

In this work, the multi-physics coupling of CTF-PARCS for both steady-state and transient simulations is evaluated focusing on the quantification of the importance of the modeling fidelity when applied to the Best Estimate Plus Uncertainty safety analysis. For the steady-state, a Picard iteration approach is adopted in the coupled code while for the transient an explicit approach has been developed as a first step. The CTF-PARCS coupling is applied to the REA at HFP BOC scenario proposed in the exercise I-2c of LWR-UAM benchmark Phase III. Two different modeling choices are investigated for the nominal calculations. The first consists of including the 3D burnup information in CTF, and the second of modeling the spacers grids in both PARCS and CTF. It is observed that due to the degradation of the fuel thermal conductivity, the 3D burnup inclusion in CTF leads to significant higher centerline temperatures than the base case, with a difference up to 300 K. The spacer grids modeling shows the expected power and temperature decrease in the axial positions of the spacer grids. For the uncertainty quantification study, the multi-physics modeling including the burnup information in CTF is used. The results show that the inserted reactivity has a relative standard deviation of 13.6% mainly attributed to the nuclear data. The maximum linear power has a relative standard deviation of 16.2% due to the nuclear data as well, while the maximum centerline temperature has a relative standard deviation of 3.5% primarily caused by the fuel thermal conductivity. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

Confronting Domain Shift in Trained Neural Networks

Neural networks (NNs) are known as universal function approximators and can interpolate nonlinear functions between observed data points. However, when the target domain for deployment shifts from the training domain and NNs must extrapolate, the results are notoriously poor. Prior work Martinez et al. (2019) has shown that NN uncertainty estimates can be used to correct binary predictions in shifted domains without retraining the model. We hypothesize that this approach can be extended to correct real-valued time series predictions. As an exemplar, we consider two mechanical systems with nonlinear dynamics. The first system consists of a spring-mass system where the stiffness changes abruptly, and the second is a real experimental system with a frictional joint that is an open challenge for structural dynamicists to model efficiently. Our experiments will test whether 1) NN uncertainty estimates can identify when the input domain has shifted from the training domain and 2) whether the information used to calculate uncertainty estimates can be used to correct the NN’s time series predictions. While the method as proposed did not significantly improve predictions, our results did show potential for modifications that could improve models’ predictions and play a role in structural health monitoring systems that directly impact public safety.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Role of the likelihood for elastic scattering uncertainty quantification

In the last decade, uncertainty quantification (UQ) for optical model potentials (OMPs) has become a focal point for nuclear reaction theory, and several competing approaches for OMP UQ have recently been developed. Here, we clarify recent efforts to compare frequentist and Bayesian approaches in the context of OMP UQ [G. B. King et al., Phys. Rev. Lett. 122, 232502 (2019)]. We replicate a portion of that OMP UQ study but use independent statistical tools. Specifically, we compare two methods for OMP parameter inference from elastic scattering data: the Levenberg-Marquardt algorithm for χ 2 minimization on one hand and Markov chain Monte Carlo (MCMC) sampling on the other. Separately, we assess the common practice of using a renormalized likelihood (χ 2 /N), N being the number of data points, instead of the canonical weighted-least-squares likelihood (χ 2 ), as a way of accounting for unknown data correlations. Here, we show that for a generic linear model and for a five-parameter OMP analysis, frequentist and uniform-prior Bayesian approaches recover the same optimum and uncertainty estimates—not systematically larger uncertainties for the Bayesian approach, as was concluded in G. B. King et al., Phys. Rev. Lett. 122, 232502 (2019). Further, we show that if an additional, near-degenerate parameter is introduced into the same OMP analysis such that the parameter posterior becomes non-Gaussian, then covariance-based estimates of uncertainty become unreliable. Finally, we show that regardless of optimization approach, if χ 2 /N is used for the likelihood, the resulting parametric uncertainties increase by $\sqrt{N}$, and that this is responsible for the conclusions drawn in the revisited study. Based on our replication results, we find that a fortuitous cancellation of unreported errors and the renormalization factor can lead to improvement in empirical coverages, as was the case in the original comparative study. We emphasize that developing and applying a realistic likelihood function is an essential task in a UQ analysis, and that several recent UQ studies that employed a renormalized likelihood (i.e., including a 1/N factor) may have yielded unrealistically large uncertainties for elastic-scattering observables. If the parameter posterior deviates from multivariate-normal, a sampling-based approach like MCMC has a clear advantage over methods that assume the Laplace approximation holds. We note that empirical coverage can serve as an important internal check for the analyst whose model or data may have additional, unaccounted-for uncertainties.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Including 238 U(n,f)/ 235 U(n,f) and 239 Pu(n,f)/ 235 U(n,f) NIFFTE fission TPC Cross-sections into the Neutron Data Standards Database

The primary purpose of this report is to document how the 238 U/ 235 U and 239 Pu/ 235 U neutron induced fission cross-section ratios, 238 U(n,f)/ 235 U(n,f) and 239 Pu(n,f)/ 235 U(n,f), respectively, measured by the NIFFTE fission Time Projection Chamber (fissionTPC) were included in the most recent database (termed GMA) underlying Neutron Data Standards (NDS) evaluations. This report shows and discusses NDS input files, and underlying assumptions regarding the uncertainty estimate and necessary for including these data. This uncertainty estimate and the resulting files were based on information provided by fissionTPC experimentalists, R.J.Casperson, N.S. Bowden, L. Snyder and K.T. Schmitt for the 238 U ratio, and by L. Snyder for the 239 Pu ratio. The fissionTPC data were included twice, by D. Neudecker and V. Pronyaev, to counter-check results and exclude possible mistakes in their inclusion. It is shown in both evaluations that including fissionTPC 239 Pu(n,f)/ 235 U(n,f) data points to a lower evaluated 239 Pu(n,f) cross section above 10 MeV than the currently released NDS data. This raises the question whether a part of a previous dataset by Tovesson et al., that was previously rejected above 13 MeV for having low values, should be included in the NDS evaluation after all. The evaluated 238 U(n,f) cross section only changes significantly close to the threshold. The impact on the 235 U(n,f) cross section is minimal. fissionTPC data reduce evaluated uncertainties on both observables by 0–12% of the GMA evaluated uncertainties. However, the currently released NDS data contain in addition to these GMA evaluated uncertainties “Unrecognized Sources of Uncertainties” (USU) of 1.2%. It needs to be further discussed within the NDS project, whether the new fissionTPC data should also reduce USU.

238U(n,f)/235U(n,f)↗
DOE OSTI2026

Active operator learning with predictive uncertainty quantification for partial differential equations

With the increased prevalence of neural operators being used to provide rapid solutions to partial differential equations (PDEs), understanding the accuracy of model predictions and the associated error levels is necessary for deploying reliable surrogate models in scientific applications. Existing uncertainty quantification (UQ) frameworks employ ensembles or Bayesian methods, which can incur substantial computational costs during both training and inference. Here, we propose a lightweight predictive UQ method tailored for Deep operator networks (DeepONets) that also generalizes to other operator networks. Numerical experiments on linear and nonlinear PDEs demonstrate that the framework’s uncertainty estimates are unbiased and provide accurate out-of-distribution uncertainty predictions with a sufficiently large training dataset. Our framework provides fast inference and uncertainty estimates that can efficiently drive outer-loop analyses that would be prohibitively expensive with conventional solvers. We demonstrate how predictive uncertainties can be used in the context of Bayesian optimization and active learning problems to yield improvements in accuracy and data-efficiency for outer-loop optimization procedures. In the active learning setup, we extend the framework to Fourier Neural Operators (FNO) and describe a generalized method for other operator networks. To enable real-time deployment, we introduce an inference strategy based on precomputed trunk outputs and a sparse placement matrix, reducing evaluation time by more than a factor of five. Our method provides a practical route to uncertainty-aware operator learning in time-sensitive settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

On the uncertainty of estimating photovoltaic soiling using nearby soiling data

The accumulation of dust on the surface of photovoltaic modules can reduce their performance and affect the cost competitiveness of this technology. This phenomenon is known as soiling and can be mitigated through appropriate corrective and/or preventive actions. In order to maximize its effectiveness, it is important to plan the soiling mitigation strategy even before the PV system is operational. This is typically done through a nearest neighbor approach, by estimating soiling using data from the nearest operational photovoltaic system. This work focuses on understanding the uncertainty related to this practice. For this purpose, the semi-variance function is used to study the dissimilarity between the soiling losses of two locations in California depending on their distance. The results show that, when the soiling loss at a nearby system is used to estimate soiling of a site, the uncertainty can be approximated to increase linearly at a rate of 0.08-0.10%/km up to 60 or 80 km. After this distance, the use of a nearest neighbor approach is no longer justified, as it produces an uncertainty as big as the average soiling loss of the sites in the dataset used in this study. In some conditions, uncertainties > 0% are found also for sites located within 25 km, meaning that even close-by systems might soil differently.

14 SOLAR ENERGY↗
DOE OSTI2023

(U) A General-Purpose Code for Correlated Sampling Using Batch Statistics with MCNP6 for Fixed-Source Problems

Correlated sampling can be used to reduce the uncertainty of a difference of tallies by taking advantage of the negative covariance term in the sandwich formula. Booth first showed how correlated sampling can be applied with batch statistics using MCNP’s tally fluctuation chart (TFC) to reduce the uncertainty of a difference of tallies in fixed-source problems. Booth presented a problem in which a 1273% uncertainty in a difference was reduced to 8% by accounting for correlations. Researchers He and Su recently studied correlated sampling using the TFC in MCNP version 5. They determined that the code did not print enough digits in the TFC tally means for accurate batch statistics in some cases. After modifying the source code, they concluded that “correlated sampling can yield a standard deviation of about one magnitude smaller than that predicted by the direct, un-correlated simulation when the changes in system response are small (say about 1%), which is equivalent to saving in CPU time by a factor of 100. Such saving [sic] becomes less significant as the change in system response becomes larger.” He and Su provided the formulas needed to apply batch statistics to compute the correlated uncertainty of a difference of tallies. In this report, we follow up on their work by providing the formulas needed to apply batch statistics to compute the correlated uncertainty of a ratio of tallies and of a difference of two tallies divided by a third tally. We extend these formulas to differences and ratios of ratios. These formulas are applied to reduce the uncertainty associated with calculating a relative sensitivity. He and Su did not investigate the accuracy of their correlated sampling uncertainty estimates. We use their test problems and evaluate the accuracy of the uncertainty estimates by comparing with results obtained from random sampling, and, in simple cases, with theoretical values of the “exact” uncertainties. We find that the uncertainties obtained from batch statistics are accurate as long as at least 100 batches are used. We present a new computer code, COSUBS (COrrelated Sampling Using Batch Statistics), that reads MCNP6 TFCs and applies correlated sampling using batch statistics for the tally combinations that the user specifies. COSUBS is a very general tool that compares all TFCs for a base case and one or two perturbed cases. It computes uncertainties for ratios if given only a base case. This report is organized as follows. The equations to apply batch statistics to the difference of random tallies are reviewed in Sec. II. Section III presents the equations for applying batch statistics to a ratio of random tallies; this is useful for computing relative sensitivities using a one-sided finite difference and the relative sensitivity using the differential operator method. Section IV presents the equations for applying batch statistics to a difference of two random tallies divided by a third; this is useful for computing a relative sensitivities using a central difference. Section V presents the equations for applying batch statistics to a difference of two ratios with four random tallies. Section VI presents the equations for applying batch statistics to a one-sided finite difference estimate of the relative sensitivity of a ratio (this uses four random tallies). Section VII presents the equations for applying batch statistics to a central difference estimate of the relative sensitivity of a ratio (this uses six random tallies). Section VIII presents the equations for applying batch statistics to a sum of random tallies. Section IX discusses how to apply batch statistics using MCNP6. Section X presents COSUBS, describing its command-line options and logic. Sections XI through XVI present numerical results for various test problems. Section XVII is a summary and conclusions. Appendix A derives the theoretical Monte Carlo tally variance given certain assumptions; these variances are used to verify the batch statistics for some of the problems. Appendix B lists the MCNP6 input for the unperturbed example problem. Appendix C presents modifications made to MCNP6.3 to support this work.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

National population mapping from sparse survey data: A hierarchical Bayesian modeling framework to account for uncertainty

Population estimates are critical for government services, development projects, and public health campaigns. Such data are typically obtained through a national population and housing census. However, population estimates can quickly become inaccurate in localized areas, particularly where migration or displacement has occurred. Some conflict-affected and resource-poor countries have not conducted a census in over 10 y. We developed a hierarchical Bayesian model to estimate population numbers in small areas based on enumeration data from sample areas and nationwide information about administrative boundaries, building locations, settlement types, and other factors related to population density. We demonstrated this model by estimating population sizes in every 10- m grid cell in Nigeria with national coverage. These gridded population estimates and areal population totals derived from them are accompanied by estimates of uncertainty based on Bayesian posterior probabilities. The model had an overall error rate of 67 people per hectare (mean of absolute residuals) or 43% (using scaled residuals) for predictions in out-of-sample survey areas (approximately 3 ha each), with increased precision expected for aggregated population totals in larger areas. This statistical approach represents a significant step toward estimating populations at high resolution with national coverage in the absence of a complete and recent census, while also providing reliable estimates of uncertainty to support informed decision making.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

Statistical patterns of theory uncertainties

A comprehensive uncertainty estimation is vital for the precision program of the LHC. While experimental uncertainties are often described by stochastic processes and well-defined nuisance parameters, theoretical uncertainties lack such a description. We study uncertainty estimates for cross-section predictions based on scale variations across a large set of processes. We find patterns similar to a stochastic origin, with accurate uncertainties for processes mediated by the strong force, but a systematic underestimate for electroweak processes. We propose an improved scheme, based on the scale variation of reference processes, which reduces outliers in the mapping from leading order to next-to-leading-order in perturbation theory.

Ghosh, Aishik↗
DOE OSTI2023

Quantifying Uncertainty in PV Energy Estimates Final Report

Uncertainty in PV energy estimates is "one of the most critical areas of lack of understanding" according to independent engineers, financiers, PV model developers, and other industry stakeholders. The primary problem is a lack of rigorous, transparent, widely accepted methods for quantifying uncertainty in energy production estimates. Uncertainty in energy production estimates arises from variability of the solar resource, inexact PV performance models and their parameters, and system reliability considerations. Uncertainty in annual energy production is frequently calculated for larger projects in order to quantify financial risk. Key statistics for energy, such as the P-values "P50" and "P90" (the annual energy values that are exceeded in future years with 50\% and 90\% probability, respectively) are used by financing institutions to calculate the repayment risk for the project. The current methods to estimate these statistics are typically proprietary, specialized, and involve significant post-processing of commercial performance model results. This black-box approach leads to inconsistent P-value estimates from different parties, which reduces investors' confidence in the results. Since the financial community bases its risk assessment on these estimates, reduced confidence increases perceived project risk, and consequently financing costs. The goal of this project was to establish a set of best practices for quantifying uncertainty in energy production estimates, including identifying what sources of uncertainty must be considered with clear definitions and metrics, determining which sources are the biggest drivers of uncertainty, and providing a computationally efficient framework for combining different sources of uncertainty that is flexible enough to accommodate substitutions of data or methods when better information is available. We engaged a wide set of stakeholders to ensure industry endorsement and adoption, and leveraged complementary projects investigating individual sources of uncertainty in great detail, as well as others' work that started down this path.

ENERGY PLANNING, POLICY, AND ECONOMY,SOLAR ENERGY↗
DOE OSTI2023

Uncertainty-driven dynamics for active learning of interatomic potentials

Machine learning (ML) models, if trained to data sets of high-fidelity quantum simulations, produce accurate and efficient interatomic potentials. Active learning (AL) is a powerful tool to iteratively generate diverse data sets. In this approach, the ML model provides an uncertainty estimate along with its prediction for each new atomic configuration. If the uncertainty estimate passes a certain threshold, then the configuration is included in the data set. Here we develop a strategy to more rapidly discover configurations that meaningfully augment the training data set. The approach, uncertainty-driven dynamics for active learning (UDD-AL), modifies the potential energy surface used in molecular dynamics simulations to favor regions of configuration space for which there is large model uncertainty. The performance of UDD-AL is demonstrated for two AL tasks: sampling the conformational space of glycine and sampling the promotion of proton transfer in acetylacetone. The method is shown to efficiently explore the chemically relevant configuration space, which may be inaccessible using regular dynamical sampling at target temperature conditions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Bayesian learning with Gaussian processes for low-dimensional representations of time-dependent nonlinear systems

This work presents a data-driven method for learning low-dimensional time-dependent physics-based surrogate models whose predictions are endowed with uncertainty estimates. We use the operator inference approach to model reduction that poses the problem of learning low-dimensional model terms as a regression of state space data and corresponding time derivatives by minimizing the residual of reduced system equations. Standard operator inference models perform well with accurate training data that are dense in time, but producing stable and accurate models when the state data are noisy and/or sparse in time remains a challenge. Another challenge is the lack of uncertainty estimation for the predictions from the operator inference models. Our approach addresses these challenges by incorporating Gaussian process surrogates into the operator inference framework to (1) probabilistically describe uncertainties in the state predictions and (2) procure analytical time derivative estimates with quantified uncertainties. The formulation leads to a generalized least-squares regression and, ultimately, reduced-order models that are described probabilistically with a closed-form expression for the posterior distribution of the operators. The resulting probabilistic surrogate model propagates uncertainties from the observed state data to reduced-order predictions. Furthermore, we demonstrate the method is effective for constructing low-dimensional models of two nonlinear partial differential equations representing a compressible flow and a nonlinear diffusion–reaction process, as well as for estimating the parameters of a low-dimensional system of nonlinear ordinary differential equations representing compartmental models in epidemiology.

Data-driven model reduction↗
DOE OSTI2022

Interpretable Uncertainty Quantification in AI for HEP

Estimating uncertainty is at the core of performing scientific measurements in HEP: a measurement is not useful without an estimate of its uncertainty. The goal of uncertainty quantification (UQ) is inextricably linked to the question, "how do we physically and statistically interpret these uncertainties?" The answer to this question depends not only on the computational task we aim to undertake, but also on the methods we use for that task. For artificial intelligence (AI) applications in HEP, there are several areas where interpretable methods for UQ are essential, including inference, simulation, and control/decision-making. There exist some methods for each of these areas, but they have not yet been demonstrated to be as trustworthy as more traditional approaches currently employed in physics (e.g., non-AI frequentist and Bayesian methods). Shedding light on the questions above requires additional understanding of the interplay of AI systems and uncertainty quantification. We briefly discuss the existing methods in each area and relate them to tasks across HEP. We then discuss recommendations for avenues to pursue to develop the necessary techniques for reliable widespread usage of AI with UQ over the next decade.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Multiphysics Simulation of the NASA SIRIUS-CAL Fuel Experiment in the Transient Test Reactor Using Griffin

After approximately 50 years, NASA is restarting efforts to develop nuclear thermal propulsion (NTP) for interplanetary missions. Building upon nuclear engine tests performed from the late 1950s to the early 1970s, the present research and testing focuses on advanced materials and fabrication methods. A number of transient tests have been performed to evaluate materials performance under high-temperature, high-flux conditions, with several more experiments in the pipeline for future testing. The measured data obtained from those tests are being used to validate the Griffin reactor multiphysics code for this particular type of application. Griffin was developed at Idaho National Laboratory (INL) using the MOOSE framework. This article describes the simulation results of the SIRIUS-CAL calibration experiment in the Transient Reactor Test Facility (TREAT). SIRIUS-CAL was the first transient test conducted on NASA fuels, and although the test was performed with a relatively low core peak power, the test specimen survived a temperature exceeding 900 K. Griffin simulations of the experiment successfully matched the reactor’s power transient after calibrating the initial control rod position to match the initial reactor period. The thermal-hydraulics model largely matches the time-dependent response of a thermocouple located within the experiment specimen to within the uncertainty estimate. However, the uncertainty range is significant and must be reduced in the future.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

Evaluating the Potential and Challenges of an Uncertainty Quantification Method for Long Short–Term Memory Models for Soil Moisture Predictions

Recently, recurrent deep networks have shown promise to harness newly available satellite–sensed data for long–term soil moisture projections. However, to be useful in forecasting, deep networks must also provide uncertainty estimates. Here we evaluated Monte Carlo dropout with an input–dependent data noise term (MCD+N), an efficient uncertainty estimation framework originally developed in computer vision, for hydrologic time series predictions. MCD+N simultaneously estimates a heteroscedastic input–dependent data noise term (a trained error model attributable to observational noise) and a network weight uncertainty term (attributable to insufficiently constrained model parameters). Although MCD+N has appealing features, many heuristic approximations were employed during its derivation, and rigorous evaluations and evidence of its asserted capability to detect dissimilarity were lacking. To address this, we provided an in–depth evaluation of the scheme's potential and limitations. We showed that for reproducing soil moisture dynamics recorded by the Soil Moisture Active Passive (SMAP) mission, MCD+N indeed gave a good estimate of predictive error, provided that we tuned a hyperparameter and used a representative training data set. The input–dependent term responded strongly to observational noise, while the model term clearly acted as a detector for physiographic dissimilarity from the training data, behaving as intended. However, when the training and test data were characteristically different, the input–dependent term could be misled, undermining its reliability. Additionally, due to the data–driven nature of the model, data noise also influences network weight uncertainty, and therefore the two uncertainty terms are correlated. Altogether, this approach has promise, but care is needed to interpret the results.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Beyond Point Estimates: Benchmarking Uncertainty Quantification Methods on the AION-1 Astronomical Foundation Model

Foundation models for astronomical surveys offer powerful learned representations that can be transferred to downstream regression tasks such as galaxy property estimation. However, point predictions alone are insufficient for scientific inference; reliable uncertainty quantification (UQ) is essential. We compare seven UQ methods on galaxy property regression using frozen AION-1 foundation-model embeddings, predicting redshift, stellar mass, stellar-population age, gas-phase metallicity, and specific star-formation rate, from Legacy Survey photometry/imaging and DESI spectra, with PROVABGS-derived labels. Distribution-free conformal methods achieve marginal coverage within $\sim$1 pp of the nominal 90% across all properties, while non-conformal baselines (Deep Ensembles, MC~Dropout) fail to calibrate reliably. Among conformal approaches, Conformalized Quantile Regression (CQR) delivers the best coverage in the bin with the poorest model predictions. More importantly, only the Locally Valid and Discriminative (LVD) framework -- particularly when operating on AION-1 embeddings -- also provides finite-sample \emph{local validity}, producing intervals that adapt to each galaxy's local prediction difficulty rather than relying on marginal guarantees alone. These results establish conformal prediction, and LVD in particular, as the preferred UQ framework for uncertainty-aware inference on foundation-model embeddings in astrophysics.

Tame-Narvaez, Karla [Fermilab] (ORCID:000000022249↗
DOE OSTI2021

Apparatus and method for optimizing quantifiable behavior in configurable devices and systems

An apparatus and method are provided to perform constrained optimization of a constrained property of an apparatus, which is complex due to having several components, and these components are configurable in real-time. The optimization is achieved by detecting values of the constrained property and a plurality of other properties of the apparatus when the apparatus is configured in a first subset of the plurality of configurations. A model is learned using the detected values of the constrained property. The model represents the constrained property and can also represent other properties as a function of the configurations. The model can also include estimated uncertainties of the constrained property in the model. Then, using the d model and the estimated uncertainties, the optimal configuration can be selected to minimize an error value (e.g., the difference between a desired value and an observed value of the at least one constrained property).

Hoffmann, Henry↗
DOE OSTI2022

Understanding the Accuracy Limitations of Quantifying Methane Emissions Using Other Test Method 33A

Researchers have utilized Other Test Method (OTM) 33A to quantify methane emissions from natural gas infrastructure. Historically, errors have been reported based on a population of measurements compared to known controlled releases of methane. These errors have been reported as 2σ errors of ±70%. However, little research has been performed on the minimum attainable uncertainty of any one measurement. We present two methods of uncertainty estimation. The first was the measurement uncertainty of the state-of-the-art equipment, which was determined to be ±3.8% of the estimate. This was determined from bootstrapped measurements compared to controlled releases. The second approach of uncertainty estimation was a modified Hollinger and Richardson (H&R) method which was developed for quantifying the uncertainty of eddy covariance measurements. Using a modified version of this method applied to OTM 33A measurements, it was determined that uncertainty of any given measurement was ±17%. Combining measurement uncertainty with that of stochasticity produced a total minimum uncertainty of 17.4%. Due to the current nature of stationary single-sensor measurements and the stochasticity of atmospheric data, such uncertainties will always be present. This is critical in understanding the transport of methane emissions and indirect measurements obtained from the natural gas industry.

Heltzel, Robert↗