SEARCH · Engineering Papers

Results for “error quantification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Quantifying the structural uncertainty of the aerosol mixing state representation in a modal model

Abstract. Aerosol mixing state is an important emergent property that affects aerosol radiative forcing and aerosol–cloud interactions, but it has not been easy to constrain this property globally. This study aims to verify the global distribution of aerosol mixing state represented by modal models. To quantify the aerosol mixing state, we used the aerosol mixing state indices for submicron aerosol based on the mixing of optically absorbing and non-absorbing species (χo), the mixing of primary carbonaceous and non-primary carbonaceous species (χc), and the mixing of hygroscopic and non-hygroscopic species (χh). To achieve a spatiotemporal comparison, we calculated the mixing state indices using output from the Community Earth System Model with the four-mode version of the Modal Aerosol Module (MAM4) and compared the results with the mixing state indices from a benchmark machine-learned model trained on high-detail particle-resolved simulations from the particle-resolved stochastic aerosol model PartMC-MOSAIC. The two methods yielded very different spatial patterns of the mixing state indices. In some regions, the yearly averaged χ value computed by the MAM4 model differed by up to 70 percentage points from the benchmark values. These errors tended to be zonally structured, with the MAM4 model predicting a more internally mixed aerosol at low latitudes and a more externally mixed aerosol at high latitudes compared to the benchmark. Our study quantifies potential model bias in simulating mixing state in different regions and provides insights into potential improvements to model process representation for a more realistic simulation of aerosols towards better quantification of radiative forcing and aerosol–cloud interactions.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1991

Quantification of vapor screen analysis

The traditionally qualitative method of vapor screening is applied to the task of gathering quantitative density measurements in supersonic flows. Quantitative density information for flow around a two-dimensional biconvex airfoil at Mach 2.8 is extracted from digitized images of vapor screens by assuming a linear relation between particle density and scattering intensity. Intensity readings are calibrated through known conditions in the free stream and at the airfoil bow shock. The experimental results are compared to densities generated by a Euler code and determined to have errors of less than ten percent.

Benedict, Lance H.↗

DOE OSTI2021

A fast multi-fidelity method with uncertainty quantification for complex data correlations: Application to vortex-induced vibrations of marine risers

Here we develop a fast multi-fidelity modeling method for very complex correlations between high- and low-fidelity data by working in modal space to extract the proper correlation function. We apply this method to infer the amplitude of motion of a flexible marine riser in cross-flow, subject to vortex-induced vibrations (VIV). VIV are driven by an absolute instability in the flow, which imposes a frequency (Strouhal) law that requires a matching with the impedance of the structure; this matching is easily achieved because of the rapid parametric variation of the added mass force. As a result, the wavenumber of the riser spatial response is within narrow bands of uncertainty. Hence, an error in wavenumber prediction can cause significant phase-related errors in the shape of the amplitude of response along the riser, rendering correlation between low- and high-fidelity data very complex. Working in modal space as outlined herein, dense data from low-fidelity data, provided by the semi-empirical computer code VIVA, can correlate in modal space with few high-fidelity data, obtained from experiments or fully-resolved CFD simulations, to correct both phase and amplitude and provide predictions that agree very well overall with the correct shape of the amplitude response. We also quantify the uncertainty in the prediction using Bayesian modeling and exploit this uncertainty to formulate an active learning strategy for the best possible location of the sensors providing the high fidelity measurements.

42 ENGINEERING↗

DOE OSTI2023

Count Every Trip: Finding the Uncertainty in Energy Estimates Made from Inferred Travel Modes

To properly inform transport policy and infrastructure changes, transportation related metrics need both measured values and uncertainties of those values. Travel monitoring smartphone apps can record people's travel behavior, but trip data quality is limited by sensor errors, user labeling rates and the accuracy of inference algorithms used for travel diary creation. We discuss the use of phone app recorded travel diary data to estimate energy consumption, and propose the use of propagation of variance to find error bars for such estimates. We define energy consumption for one trip as trip length times the energy intensity per distance unit of the travel mode used. We characterize trip length errors with relative error and inferred trip mode errors with confusion matrix columns. The resulting variances of each measurement are then propagated to the final calculated energy consumption. We tested our uncertainty methods on a dataset that used phone app data combined with prompted recall, consisting of 92,234 labeled trips for over 500,000 miles. Accounting for uncertainty using expected energy intensities and variance propagation gives a dataset-wide aggregate energy consumption percent error of about 9%, within one standard deviation from the truth. Future work could involve applying similar methods to other travel diary based metrics.

ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2024

Investigation of Ethane Dehydrogenation and Hydrogenolysis on Pt(111), Pt(211), and Pt(100): Bayesian Quantification and Correction of DFT-Based Enthalpic and Entropic Uncertainties

Computational investigations of heterogeneously catalyzed reactions using density functional theory (DFT) are often inaccurate, largely due to uncertainties in the choice of DFT functional (enthalpic uncertainty) and approximations for modeling adsorbate movement along the catalyst surface (entropic uncertainty). This work illustrates that both uncertainties are significant in the investigation of ethane dehydrogenation (EDH) and hydrogenolysis on Pt catalysts by considering the complete deconstruction of ethane on Pt(111), Pt(211), and Pt(100) using microkinetic modeling (MKM). Hence, this work uses both noncalibrated and Bayesian-calibrated MKMs to quantify and correct inaccuracies in macroscopic properties due to both uncertainties. A Bayesian approach to the correction of entropic errors was introduced using a “Modified Fermi Function (MFF)” to calibrate between the two bounds of entropy represented by the harmonic oscillator (HO) and free translator (FT) approximations. Regardless of enthalpic and entropic uncertainties, all three surfaces are capable of ethane activation; however, Pt(211) was found to be the most active and is largely responsible for methane production. Next, Pt(111) is largely responsible for acetylene production, and Pt(100) has the highest ethylene selectivity but is most susceptible to coking. By comparison of different calibrated models, the FT entropy approximation was found to better describe EDH under typical experimental conditions. Statistical evidence was found to support Pt(111) as the active site for EDH, assuming that one single site is responsible for the chemistry. On the three surfaces, competing second dehydrogenations to CH 2 CH 2 and CH 3 CH were observed as well as isomerization of CH 3 CH back to CH 2 CH 2 and deeper dehydrogenation of CH 3 CH. In conclusion, C–C cleavage was found to largely proceed via the CH 3 C intermediate on Pt(100) and Pt(111), while on Pt(211), it was via both CHC and CH 3 C.

Bayesian model selection↗

DOE OSTI2021

Chemometrics and Experimental Design for the Quantification of Nitrate Salts in Nitric Acid: Near-Infrared Spectroscopy Absorption Analysis

Implementing remote, real-time spectroscopic monitoring of radiochemical processing streams in hot cell environments requires efficiency and simplicity. The success of optical spectroscopy for the quantification of species in chemical systems highly depends on representative training sets and suitable validation sets. Selecting a training set (i.e., calibration standards) to build multivariate regression models is both time- and resource-consuming using standard one-factor-at-a-time approaches. This study describes the use of experimental design to generate spectral training sets and a validation set for the quantification of sodium nitrate (0–1 M) and nitric acid (0.1–10 M) using the near-infrared water band centered at 1440 nm. Partial least squares regression models were built from training sets generated by both D- and I-optimal experimental designs and a one-factor-at-a-time approach. The prediction performance of each model was evaluated by comparing the bias and standard error of prediction for statistical significance. D- and I-optimal designs reduced the number of samples required to build regression models compared with one-factor-at-a-time while also improving performance. Models must be confirmed against a validation sample set when minimizing the number of samples in the training set. The D-optimal design performed the best when considering both performance and efficiency by improving predictive capability and reducing number of samples in the training set by 64% compared with the one-factor-at-a-time approach. The experimental design approach objectively selects calibration and validation spectral data sets based on statistical criterion to optimize performance and minimize resources.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

DOE OSTI2025

Partnership Center for High-Fidelity Boundary Plasma Simulation (Final Report)

Within the Partnership Center for High-Fidelity Boundary Plasma Simulation (HBPS), work at UT-Austin was aimed at improved verification, validation, and uncertainty quantification (VVUQ) for edge plasma simulations and on performing gyrokinetics simulations of pedestal instabilities and turbulence in order to expand foundational understanding of pedestal transport. Regarding VVUQ, the accomplishments can be summarized as follows. First, it was shown that the Moment Preserving Constrained Resampling technique, when applied periodically in particle-in-cell simulations in the XGC code, can dramatically improve the accuracy of the simulation at essentially equivalent computational cost. Second, a technique for estimating model correlations, which are required to solve the model selection and sample allocation problem in multifidelity UQ techniques, without sampling the highest fidelity, most computationally expensive model, was developed and demonstrated. Third, previously developed methods for estimating statistical and discretization errors were applied to numerical methods relevant to edge plasma simulations, namely in particle-in-cell-based approaches, and shown to work. Finally, benchmark studies for comparing gyrokinetic codes were developed and performed, leading to reasonable agreement between four commonly used codes. Regarding physics studies, gyrokinetic simulations to investigate microtearing modes in the DIII-D pedestal were performed using the GENE code.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2023

Count Every Trip: Finding the Uncertainty in Energy Estimates Made from Inferred Travel Modes

ADVANCED PROPULSION SYSTEMS,MATHEMATICS AND COMPUT↗

DOE OSTI2023

Uncertain of uncertainties? A comparison of uncertainty quantification metrics for chemical data sets

Abstract With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet to be established and different studies on uncertainties generally uses different metrics to evaluate them. We compare three of the most popular validation metrics (Spearman’s rank correlation coefficient, the negative log likelihood (NLL) and the miscalibration area) to the error-based calibration introduced by Levi et al. ( Sensors 2022 , 22 , 5540). Importantly, metrics such as the negative log likelihood (NLL) and Spearman’s rank correlation coefficient bear little information in themselves. We therefore introduce reference values obtained through errors simulated directly from the uncertainty distribution. The different metrics target different properties and we show how to interpret them, but we generally find the best overall validation to be done based on the error-based calibration plot introduced by Levi et al. Finally, we illustrate the sensitivity of ranking-based methods (e.g. Spearman’s rank correlation coefficient) towards test set design by using the same toy model ferent test sets and obtaining vastly different metrics (0.05 vs. 0.65).

Rasmussen, Maria H.↗

NASA NTRS2012

Uncertainty Quantification for Polynomial Systems via Bernstein Expansions

This paper presents a unifying framework to uncertainty quantification for systems having polynomial response metrics that depend on both aleatory and epistemic uncertainties. The approach proposed, which is based on the Bernstein expansions of polynomials, enables bounding the range of moments and failure probabilities of response metrics as well as finding supersets of the extreme epistemic realizations where the limits of such ranges occur. These bounds and supersets, whose analytical structure renders them free of approximation error, can be made arbitrarily tight with additional computational effort. Furthermore, this framework enables determining the importance of particular uncertain parameters according to the extent to which they affect the first two moments of response metrics and failure probabilities. This analysis enables determining the parameters that should be considered uncertain as well as those that can be assumed to be constants without incurring significant error. The analytical nature of the approach eliminates the numerical error that characterizes the sampling-based techniques commonly used to propagate aleatory uncertainties as well as the possibility of under predicting the range of the statistic of interest that may result from searching for the best- and worstcase epistemic values via nonlinear optimization or sampling.

Crespo, Luis G.↗

DOE OSTI2020

Three dimensional cluster analysis for atom probe tomography using Ripley’s K-function and machine learning

The size and structure of spatial molecular and atomic clustering can significantly impact material properties and is therefore important to accurately quantify. Ripley’s K-function (K(r)), a measure of spatial correlation, can be used to perform such quantification when the material system of interest can be represented as a marked point pattern. This work demonstrates how machine learning models based on K (r)-derived metrics can accurately estimate cluster size and intra-cluster density in simulated three dimensional (3D) point patterns containing spherical clusters of varying size; over 90% of model estimates for cluster size and intra-cluster density fall within 11% and 18% error of the true values, respectively. These K (r)-based size and density estimates are then applied to an experimental APT reconstruction to characterize MgZn clusters in a 7000 series aluminum alloy. Here we find that the estimates are more accurate, consistent, and robust to user interaction than estimates from the popular maximum separation algorithm. Using K (r) and machine learning to measure clustering is an accurate and repeatable way to quantify this important material attribute.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Machine learning surrogate of physics-based building-stock simulator for end-use load forecasting

Building energy models are used to simulate heat and mass transfer and estimate end-use load in buildings. With the proliferation of solar photovoltaics on residential and commercial buildings, increasingly, buildings are expected to provide grid services, for which accurate and computationally efficient building energy simulations and end-use load prediction are imperative. Existing building energy simulation tools, however, have significant computational overhead that make them less practical in real-time deployment for optimization, design, uncertainty quantification and control in building energy management systems. Here this article presents a data-driven machine learning model based on light gradient boosting method (LightGBM) as a surrogate for a physics-based simulator for residential buildings to predict end-use load. The machine learning based surrogate model accounts for time-series related variables, seasonality and trend component of end-use load, and history of end-use load. The accuracy of the surrogate model is assessed on the prediction of the load profiles of 100 different houses in Cook County, Illinois, USA. The LightGBM surrogate model is shown to reduce the root-mean-squared error by 53% relative to a reference decision tree (DT) based model reported previously in the literature. Moreover, the model predicts the load spikes and high-ramp rate events throughout the year which are often the Achilles heel of other models in the literature. The machine learning based surrogate model is demonstrated to be computationally efficient, with a ten-fold reduction in the computational time compared to a physics-based building energy simulation, and suitable for uncertainty analysis and real-time control of building characteristics in response to uncertainty.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2023

Adaptive learning-driven high-throughput synthesis of oxygen reduction reaction Fe–N–C electrocatalysts

Reducing human reliance on inefficient energy systems and fossil fuels has become more urgent due to the consequences of global climate change. However, traditional trial-and-error approaches have hampered our ability to accelerate the discovery and implementation of functional materials for efficient energy conversion devices, such as polymer electrolyte fuel cells (PEFCs). To address this, we develop an adaptive learning framework that integrates machine learning and state-of-the-art capabilities in high-throughput synthesis to achieve expedited optimization of iron-nitrogen-carbon PEFC oxygen reduction reaction (ORR) electrocatalysts. We use statistical inference, uncertainty quantification, and global optimization to build a computational design-of-experiment tool that identifies the optimum compositions to be investigated next to reduce the demands placed on experimental materials discovery. We benchmark the ability of the proposed strategy to discover optimum catalyst synthesis conditions in a six-dimensional search space when starting with a thirty-six-sample database. By following the adaptive learning strategy, we synthesize fourteen new catalysts from approximately ten billion unique compositions and discover four catalysts that outperform all original samples. The best machine learning-optimized catalyst is 33% more active than the highest-performing one in the initial database, showing an ORR activity seven times larger than those typically reported for the same class of materials.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Bayesian Optimized Deep Ensemble for Uncertainty Quantification of Deep Neural Networks: a System Safety Case Study on Sodium Fast Reactor Thermal Stratification Modeling

Deep neural networks (DNNs) are increasingly important to scientific computing and engineering system simulations. Accurate uncertainty quantification (UQ) for DNNs is critical in safety-sensitive engineering domains. Traditional Deep Ensemble (DE) methods, while easy to implement, frequently suffer from poorly calibrated uncertainty estimates and limited predictive accuracy due to reliance on fixed architectures with varied weight initializations. To address these issues, we introduce a workflow that combines Bayesian Optimization (BO) and DE. The workflow is modular, scalable, and integrates parallel BO initialized with Sobol sequences to individually optimize the hyperparameters of each ensemble member. This method enhances ensemble diversity, improves predictive accuracy, and provides reliable uncertainty estimates. We evaluate the proposed BODE approach in a sodium fast reactor thermal stratification modeling case study, where we used a densely connected convolutional neural network to predict turbulent viscosity during the reactor transient with consideration of data noise. We benchmark its performance against several optimization approaches, including baseline deep ensemble, evolutionary algorithm-optimized ensemble, ensemble formed via random search combined with greedy selection, and a BO ensemble using random initialization. Here, our results demonstrate superior performance of the developed BODE approach. In noise-free scenarios, BODE notably reduces incorrect aleatoric uncertainty and significantly enhances predictive accuracy. Under conditions of 5% and 10% Gaussian noise, BODE adaptively quantifies uncertainty proportional to data noise, achieving up to an 80% reduction in root mean square error compared to baseline methods and producing well-calibrated prediction intervals.

Bayesian optimization↗

DOE OSTI2023

Estimating Travel Energy Consumption Uncertainty Based on Inferred Travel Mode and Sensed Travel Length

To properly inform transport policy and infrastructure changes, transportation related metrics need both measured values and uncertainties of those values. Travel monitoring smartphone apps can record people's travel behavior, but trip data quality is limited by sensor errors, user labeling rates and the accuracy of inference algorithms used for travel diary creation. We discuss the use of phone app recorded travel diary data to estimate energy consumption, and propose the use of propagation of variance to find error bars for such estimates. We define energy consumption for one trip as trip length times the energy intensity per distance unit of the travel mode used. We characterize trip length errors with relative error and inferred trip mode errors with confusion matrix columns. The resulting variances of each measurement are then propagated to the final calculated energy consumption. We tested our uncertainty methods on a dataset that used phone app data combined with prompted recall, consisting of 92,234 labeled trips for over 500,000 miles. Accounting for uncertainty using expected energy intensities and variance propagation gives a dataset-wide aggregate energy consumption percent error of about 8%, within one standard deviation from the truth. Future work could involve applying similar methods to other travel diary based metrics.

ADVANCED PROPULSION SYSTEMS,ENERGY PLANNING, POLIC↗

DOE OSTI2024

A First-Principles Approach to Modeling Surface Site Stabilities on Multimetallic Catalysts

The study of multimetallic alloys and the multitude of possible surface compositions have sparked a tremendous interest in engineering low-cost materials with high activity and selectivity in heterogeneous catalysis. Multimetallic systems provide complementary functionalities and an unprecedented tunability when designing catalyst formulations. However, due to their immense structural and compositional complexity, the investigation and identification of an optimal catalyst is a tedious and time-consuming process, both experimentally and theoretically. Therefore, theoretical design principles are highly desirable to accelerate the screening of catalyst structures across the vast compositional space. In this paper, we introduce a simple and general model for predicting the site stability of multimetallic surfaces and nanoparticles, which is based on physical principles. The model requires only a small set of density functional theory (DFT) calculations of metal atom binding energies on monometallic and dilute alloy surface slabs to optimize the parameters in the simple model. The resulting model allows for the quantification of the stability of any particular atom site in any conceivable chemical environment across a wide range of morphologies, sizes, and arrangements by interpolating the derived parameters from a monometallic system to a completely diluted alloyed system. Herein, we demonstrate the robustness of the model across an extensive data set of transition metal alloy surfaces and 147-atoms cuboctahedral nanoparticles (NPs) composed of IrRhRu and PtPdRu. In conclusion, our approach yields mean absolute errors of ≈0.15 (IrRhRu), 0.20 (PtPdRu), 0.19 (IrRhRu NP), and 0.26 (PtPdRu NP) eV relative to site binding energies calculated using DFT.

36 MATERIALS SCIENCE↗

NASA NTRS2019

Validation of ICESat-2 ATLAS Bathymetry and Analysis of ATLAS’s Bathymetric Mapping Performance

NASA’s Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) was launched in September, 2018. The satellite carries a single instrument, ATLAS (Advanced Topographic Laser Altimeter System), a green wavelength, photon-counting lidar, enabling global measurement and monitoring of elevation with a primary focus on the cryosphere. Although bathymetric mapping was not one of the design goals for ATLAS, pre-launch work by our research team showed the potential to map bathymetry with ICESat-2, using data from MABEL (Multiple Altimeter Beam Experimental Lidar), NASA’s high-altitude airborne ATLAS emulator, and adapting the laser-radar equation for ATLAS specific parameters. However, many of the sensor variables were only approximations, which limited a full assessment of the bathymetric mapping capabilities of ICESat-2 during pre-launch studies. Following the successful launch, preliminary analyses of the geolocated photon returns have been conducted for a number of coastal sites, revealing several salient examples of seafloor detection in water depths of up to ~40 m. The geolocated seafloor photon returns cannot be taken as bathymetric measurements, however, since the algorithm used to generate them is not designed to account for the refraction that occurs at the air–water interface or the corresponding change in the speed of light in the water column. This paper presents the first early on-orbit validation of ICESat-2 bathymetry and quantification of the bathymetric mapping performance of ATLAS using data acquired over St. Thomas, U.S. Virgin Islands. A refraction correction, developed and tested in this work, is applied, after which the ICESat-2 bathymetry is compared against high-accuracy airborne topo-bathymetric lidar reference data collected by the U.S. Geological Survey (USGS) and the National Oceanic and Atmospheric Administration (NOAA). The results show agreement to within 0.43—0.60 m root mean square error (RMSE) over 1 m grid resolution for these early on-orbit data. Refraction-corrected bottom return photons are then inspected for four coastal locations around the globe in relation to Visible Infrared Imaging Radiometer Suite (VIIRS) K(d)(490) data to empirically determine the maximum depth mapping capability of ATLAS as a function of water clarity. It is demonstrated that ATLAS has a maximum depth mapping capability of nearly 1 Secchi in depth for water depths up to 38 m and K(d)(490) in the range of 0.05–0.12 per m. Collectively, these results indicate the great potential for bathymetric mapping with ICESat-2, offering a promising new tool to assist in filling the global void in nearshore bathymetry.

ICESat-2↗

NASA NTRS1981

Quantification of airport community noise impact in terms of noise levels, population density, and human subjective response

The Fraction Impact Method (FIM), developed by the National Research Council (NRC) for assessing the amount and physiological effect of noise, is described. Here, the number of people exposed to a given level of noise is multiplied by a weighting factor that depends on noise level. It is pointed out that the Aircraft-noise Levels and Annoyance MOdel (ALAMO), recently developed at NASA Langley Research Center, can perform the NRC fractional impact calculations for given modes of operation at any U.S. airport. The sensitivity of these calculations to errors in estimates of population, noise level, and human subjective response is discussed. It is found that a change in source noise causes a substantially smaller change in contour area than would be predicted simply on the basis of inverse square law considerations. Another finding is that the impact calculations are generally less sensitive to source noise errors than to systematic errors in population or subjective response.

Deloach, R.↗