Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “uncertainty estimation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
NASA NTRS2006

Assessment of MODIS-Derived Visible and Near-IR Aerosol Optical Properties and their Spatial Variability in the Presence of Mineral Dust

Mineral dust aerosol is among the most difficult aerosol species to measure quantitatively from space. In this paper, we evaluate MODIS retrievals of spectral aerosol optical depth (AOD) from the visible to the near-IR off the US West Coast using measurements taken by the NASA Ames Airborne Tracking Sunphotometer, AATS-14, during the EVE (Extended-MODIS-lambda Validation Experiment, 2004) campaign in April of 2004. In EVE, a total of 35 and 49 coincident over-ocean suborbital measurements at the nominal level-2 retrieval scale of 10 km x 10 km were collected for Terra and Aqua, respectively. For MODIS-Terra about 80% of the AOD retrievals are within the estimated uncertainty, DELTA tau = plus or minus 0.03 plus or minus 0.05 tau; this is true for both the visible (here defined to include 466-855 nm) and near-IR (here defined to include 1243-2119 nm) retrievals. For MODIS-Aqua about 45% of the AOD retrievals are within DELTA tau = plus or minus 0.03 plus or minus 0.05 tau; the fraction of near-IR retrievals that fall within this uncertainty range is about 27%. We found an rms difference of 0.71 between the sunphotometer snd MODIS-Aqua estimates of the visible (553-855 nm) Angstrom exponent, while the MODIS-Terra visible Angstrom exponents show an rms difference of only 0.29 when compared to AATS. The cause of the differences in performance between MODIS-Terra and MODIS-Aqua could be instrument calibration and needs to be explored further. The spatial variability of AOD between retrieval boxes as derived by MODIS is generally larger than that indicated by the sunphotometer data.

Redemann, J.↗
NASA NTRS2007

Technical Evaluation Report for Symposium AVT-147: Computational Uncertainty in Military Vehicle Design

The complexity of modern military systems, as well as the cost and difficulty associated with experimentally verifying system and subsystem design makes the use of high-fidelity based simulation a future alternative for design and development. The predictive ability of such simulations such as computational fluid dynamics (CFD) and computational structural mechanics (CSM) have matured significantly. However, for numerical simulations to be used with confidence in design and development, quantitative measures of uncertainty must be available. The AVT 147 Symposium has been established to compile state-of-the art methods of assessing computational uncertainty, to identify future research and development needs associated with these methods, and to present examples of how these needs are being addressed and how the methods are being applied. Papers were solicited that address uncertainty estimation associated with high fidelity, physics-based simulations. The solicitation included papers that identify sources of error and uncertainty in numerical simulation from either the industry perspective or from the disciplinary or cross-disciplinary research perspective. Examples of the industry perspective were to include how computational uncertainty methods are used to reduce system risk in various stages of design or development.

Radespiel, Rolf↗
NASA NTRS2018

Physics-Based Approach to Predict the Solar Activity Cycles

Observations of the complex highly non-linear dynamics of global turbulent flows and magnetic fields are currently available only from Earth-side observations. Recent progress in helioseismology has provided us some additional information about the subsurface dynamics, but its relation to the magnetic field evolution is not yet understood. These limitations cause uncertainties that are difficult take into account, and perform proper calibration of dynamo models. The current dynamo models have also uncertainties due to the complicated turbulent physics of magnetic field generation, transport and dissipation. Because of the uncertainties in both observations and theory, the data assimilation approach is natural way for the solar cycle prediction and estimating uncertainties of this prediction. I will discuss the prediction results for the upcoming Solar Cycle 25 and their uncertainties and affect of Ensemble Kalman Filter parameters to resulting predictions.

Physics-Based↗
DOE OSTI2020

Equations of state, phase relations, and oxygen fugacity of the Ru-RuO 2 buffer at high pressures and temperatures

Experimental studies and measurements of inclusions in diamonds show that ferric iron components are increasingly stabilized with depth in the mantle. To determine the thermodynamic stability of such components, their concentration needs to be measured at known oxygen fugacities. The metal-oxide pair Ru and RuO 2 are ideal as an internal oxygen fugacity buffer in high-pressure experiments. Both phases remain solid to high temperatures and react minimally with silicates, only exchanging oxygen. To calculate oxygen fugacities at high pressure and temperature, however, requires information on the phase relations and equation of state properties of the solid phases. We have made in situ synchrotron X-ray diffraction measurements in a multi-anvil press on mixtures of Ru and RuO 2 to 19.4 GPa and 1473 K with which we have determined phase relations of the RuO 2 phases and derived thermal equations of state (EoS) parameters for both Ru and RuO 2 . Rutile-structured RuO 2 was found to undergo two phase transformations, first at ~7 GPa to an orthorhombic structure and then above 12 GPa to a cubic structure. The phase boundary of the cubic phase was constrained for the first time at high pressure and temperature. We have derived a continuous Gibbs free energy expression for the tetragonal and orthorhombic phases of RuO 2 by fitting the second-order phase transition boundary and P-V-T data for both phases, using a model based on Landau theory. The transition between the orthorhombic and cubic phases was then used along with EoS terms derived for both phases to determine a Gibbs free energy expression for the cubic phase. We have used these data to calculate the oxygen fugacity of the Ru + O 2 = RuO 2 equilibrium, which we have parameterized as a single polynomial across the stability fields of all three phases of RuO 2 . The expression is log 10 f O 2 (Ru – RuO 2 ) = (7.782 – 0.00996P + 0.001932P 2 – 3.76 × 10 –5 P 3 ) + (–13 763 + 592P – 3.955P 2 )/T + (–1.05 × 10 6 – 4622P)/T 2 , which should be valid from room pressure up to 25 GPa and 773–2500 K, with an estimated uncertainty of 0.2 log units. Our calculated fO 2 is shown to be up to 1 log unit lower than estimates that use previous expressions or ignore EoS terms.

Armstrong, Katherine↗
NASA NTRS2004

Path following using visual odometry for Mars Rover in high-slip environments.

An architecture for autonomous operation of Mars rovers in high slip environments has been designed, implemented, and tested. This architecture is composed of several key technologies that enable the rover to accurately follow a designated path, compensate for slippage, and reach intended goals independent of the terrain over which it is traversing (within the mechanical constraints of the mobility system). These technologies include: visual odometry, full vehicle kinematics, a Kalman filter, and a slop compensation/path follower. Visual odometry tracks distinctive scene features in stereo imagery to estimate rover motion between successively acquired stereo image pairs using a maximum likelihood motion estimation algorithm. The full vehicle kinematics for a rocker-bogie suspension system estimates motion, with a no-slip assumption, by measuring wheel, rates, and rocker, bogie, and steering angles. The Kalman filter merges data from an Inertial Measurement Unit (IMU) and visual odometry. This merged estimate is then compared to the kinematic estimate to determining (tracking into account estimate uncertainties) if and how much slippage has occurred then a slip vector is calculated by differencing the current Kalman filter estimate from the kinematic estimate. This slip vector is then used, in conjunction with the inverse kinematics, to determine the necessary wheel velocities and steering angles to compensate for slip and follow the desired path.

Roumeliotis, Stergios I.↗
NASA NTRS1977

Flash photolysis resonance fluorescence investigation of the reaction of Cl/2P/ atoms with ClONO2

Rate constants for the removal of Cl(2P) atoms by homogeneous gas phase reaction with ClONO2 were measured using the flash photolysis resonance fluorescence technique at 224, 250, and 273 K. The values obtained have been used to derive the above Arrhenius equation. The uncertainties expressed are 95% confidence limits obtained using the statistical standard deviation of the individual rate constants plus a 15% additional estimated uncertainty attributed to difficulties in chlorine nitrate concentration measurements. The results indicate that destruction of chlorine nitrate by reaction with chlorine atoms is perhaps the least important of the atmospheric reactive channels, and is at least three orders of magnitude slower than calculated photodissociation rates at all stratospheric altitudes.

Kurylo, M. J.↗
DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗
DOE OSTI2022

Uncertainty-aware mixed-variable machine learning for materials design

Abstract Data-driven design shows the promise of accelerating materials discovery but is challenging due to the prohibitive cost of searching the vast design space of chemistry, structure, and synthesis methods. Bayesian optimization (BO) employs uncertainty-aware machine learning models to select promising designs to evaluate, hence reducing the cost. However, BO with mixed numerical and categorical variables, which is of particular interest in materials design, has not been well studied. In this work, we survey frequentist and Bayesian approaches to uncertainty quantification of machine learning with mixed variables. We then conduct a systematic comparative study of their performances in BO using a popular representative model from each group, the random forest-based Lolo model (frequentist) and the latent variable Gaussian process model (Bayesian). We examine the efficacy of the two models in the optimization of mathematical functions, as well as properties of structural and functional materials, where we observe performance differences as related to problem dimensionality and complexity. By investigating the machine learning models’ predictive and uncertainty estimation capabilities, we provide interpretations of the observed performance differences. Our results provide practical guidance on choosing between frequentist and Bayesian uncertainty-aware machine learning models for mixed-variable BO in materials design.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Probabilistic Prediction of Geomagnetic Storms and the K p index

Geomagnetic activity is often described using summary indices to summarize the likelihood of space weather impacts, as well as when parameterizing space weather models. The geomagnetic index K p in particular, is widely used for these purposes. Current state-of-the-art forecast models provide deterministic K p predictions using a variety of methods – including empirically-derived functions, physics-based models, and neural networks – but do not provide uncertainty estimates associated with the forecast. This paper provides a sample methodology to generate a 3-hour-ahead K p prediction with uncertainty bounds and from this provide a probabilistic geomagnetic storm forecast. Specifically, we have used a two-layered architecture to separately predict storm (K p ≥ 5 – ) and non-storm cases. As solar wind-driven models are limited in their ability to predict the onset of transient-driven activity we also introduce a model variant using solar X-ray flux to assess whether simple models including proxies for solar activity can improve the predictions of geomagnetic storm activity with lead times longer than the L1-to-Earth propagation time. By comparing the performance of these models we show that including operationally-available information about solar irradiance enhances the ability of predictive models to capture the onset of geomagnetic storms and that this can be achieved while also enabling probabilistic forecasts.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Ab Initio Limits of Atomic Nuclei

We predict the limits of existence of atomic nuclei, the proton and neutron drip lines, from the light through medium-mass regions. Starting from a chiral two- and three-nucleon interaction with good saturation properties, we use the valence-space in-medium similarity renormalization group to calculate ground-state and separation energies from helium to iron, nearly 700 isotopes in total. Here, we use the available experimental data to quantify the theoretical uncertainties for our ab initio calculations towards the drip lines. Where the drip lines are known experimentally, our predictions are consistent within the estimated uncertainty. For the neutron-rich sodium to chromium isotopes, we provide predictions to be tested at rare-isotope beam facilities.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2023

Uncertain of uncertainties? A comparison of uncertainty quantification metrics for chemical data sets

Abstract With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet to be established and different studies on uncertainties generally uses different metrics to evaluate them. We compare three of the most popular validation metrics (Spearman’s rank correlation coefficient, the negative log likelihood (NLL) and the miscalibration area) to the error-based calibration introduced by Levi et al. ( Sensors 2022 , 22 , 5540). Importantly, metrics such as the negative log likelihood (NLL) and Spearman’s rank correlation coefficient bear little information in themselves. We therefore introduce reference values obtained through errors simulated directly from the uncertainty distribution. The different metrics target different properties and we show how to interpret them, but we generally find the best overall validation to be done based on the error-based calibration plot introduced by Levi et al. Finally, we illustrate the sensitivity of ranking-based methods (e.g. Spearman’s rank correlation coefficient) towards test set design by using the same toy model ferent test sets and obtaining vastly different metrics (0.05 vs. 0.65).

Rasmussen, Maria H.↗
NASA NTRS1999

Satellite Data Assimilation

Satellite altimetry provides the only routine observation of a dynamic variable of the global ocean. For instance, measurements from the TOPEX/POSEIDON altimeter have provided dramatic basin-wide images of the space-time evolution of sea level associated with the 1997-98 El Nino event. The nature of large-scale ocean circulation is being studied that underlies such sea level changes measured by satellite altimetry. To first approximation, properties of large-scale (order 1000 km and larger) sea level variability is latitudinally dependent. An analysis using a general circulation model shows that sea level changes are largely due to wind-driven baroclinic (depth dependent) circulation in the tropics (within 20 degrees of the equator), but are primarily due to the expansion and contraction of near surface water in temperate latitudes (between 20 deg. and 40 deg.) forced by seasonal heating and cooling. In contrast, wind-driven barotropic (depth independent) circulation dominates sea level variability in high latitudes (40 deg. and higher) characterized by periods that are as short as a few days. The presence of such inhomogeneity and the significance of high-frequency, large-scale sea level changes had not been fully recognized prior to this study, and are summarized in Fukumori et al. Numerical models provide theoretical relationships among properties that can be inverted using observations so as to estimate the entire state of the ocean, including properties that are otherwise difficult to measure remotely. The process is data assimilation. An approximate Kalman filter and smoother have been devised to assimilate three years of TOPEX/POSEIDON sea level data into a global ocean general circulation model. The figure below demonstrates the skill of the assimilation, and shows the altimeter assimilated estimate being in closer agreement than the simulation is with independent in situ measurements of subsur ace temperature and velocity, consistent with formal uncertainty estimates. Results from this study have been summarized and submitted for publication. The study demonstrates the feasibility of global ocean data assimilation and illustrates applications in monitoring and understanding of processes controlling the evolution of the ocean. Additional information is contained in the original.

Fukumori, Ichiro↗
DOE OSTI2021

Geant4Reweight: a framework for evaluating and propagating hadronic interaction uncertainties in Geant4

Geant4Reweight is an open-source C++ framework that allows users to weight tracks produced by the Geant4 particle transport Monte Carlo simulation according to hadron interaction cross section variations and estimate uncertainties in Geant4 interaction models by comparing the simulation’s hadron interaction cross section predictions to data. The ability to weight hadron transport as simulated by Geant4 is crucial to the propagation of systematic uncertainties related to secondary hadronic interactions in current and upcoming neutrino oscillation experiments, including MicroBooNE, NOvA, and DUNE, aswell as hadron test beam experiments such as ProtoDUNE. Here, we provide motivation for weighting hadron tracks in Geant4 in the context of systematic uncertainty propagation, a description of Geant4’s transport simulation technique, and a description of our weighting technique and fitting framework in the momentum range 0–10 GeV/c, which is typical for the hadrons produced by neutrino interactions in these experiments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Robust calibration of non-perturbative models with history matching

We apply, for the first time, Bayes Linear Emulation and History Matching to the calibration of non-perturbative models in Monte Carlo event generators. In contrast to the usual approach of "Monte Carlo tuning", History Matching does not result in best-fit plus ellipsoidal parameter uncertainty estimates but instead identifies all parameter space regions that are consistent with data. This approach leads to a systematic and robust quantification of parametric uncertainties in the models, especially in those challenging cases where different, possibly disjoint, regions of parameter space deliver similar results, which are usually not properly treated with current methodology. We highlight the power of this method with the hadronisation models available through Sherpa : the built-in cluster fragmentation Ahadic and string fragmentation through an interface to Pythia .

Iskauskas, Andrew [Durham U. (main)] (ORCID:000000↗
DOE OSTI2023

Multi-Fidelity Gaussian Process for Distribution System Voltage Probabilistic Estimation with PVs

The increasing penetration of behind-the-meter PVs causes challenges to maintain voltage security due to the lack of distribution system visibility. This paper proposes a nonlinear autoregressive Gaussian process (NARGP) approach to fuse limited number of SCADA/AMI data together with historical pseudo measurements for distribution node voltage probabilistic estimation. The high-fidelity SCADA data are fused with the low-fidelity AMI and pseudo measurements by the autoregressive algorithm embedded in the Gaussian process. This allows us to use multi-fidelity data to achieve entire distribution system voltage visibility. Numerical results carried out on the IEEE 123node system demonstrate that the NARGP method is able to obtain high accuracy in estimating bus voltage and quantifying estimation uncertainties as compared to other approaches.

distribution system estimation↗
DOE OSTI2020

Discovering the nature of neutrinos: How nuclear theory will advance this grand endeavor. Final report

Why is there more matter than antimatter in the visible universe? Are neutrinos their own antiparticles? These are some of the most profound questions in science. The detection of a (0vββ) neutrinoless doublebeta decay and the measurement of its half-life is one of the most important experimental endeavors in science because it will answer these questions. Paramount to these measurements are accurate predictions with quantified uncertainties of the 0vββ-decay nuclear transitions, especially for the germanium-76 and xeon-136 nuclei, two candidate materials for building the next generation of DOE funded 0vββ-decay experiments. These predictions will guide the Office of Science’s down selection of experiments as well as aid in determining how much material is needed by these experiments. We studied a specific difficult to assess source of uncertainty that will impact coming microscopic predictions, uncertainty which arises from an incomplete yet systematically improvable model of interactions between nucleons and of nucleons with the other particles involved in decay process, referred to as the theory uncertainty. We employed a surrogate model of the germanium-76 nucleus, and its decay products, just detailed enough to adequately mimic the decaying nucleons need for a reasonable measure of this uncertainty and nothing more, avoiding the immense computational expense required in quantifying uncertainties of the full calculation. We found that microscopic calculations of this process can be expected to achieve relative theory uncertainties around 30%, including an unconstrained contribution that had the potential to spoil microscopic calculations. This results in a nearly order of magnitude on previously estimated uncertainties for this rare decay process.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS2011

Evaluation and Windspeed Dependence of MODIS Aerosol Retrievals Over Open Ocean

The Maritime Aerosol Network (MAN) data set provides high quality ground-truth to validate the MODIS aerosol product over open ocean. Prior validation of the ocean aerosol product has been limited to coastal and island sites. Comparing MODIS Collection 5 ocean aerosol retrieval products with collocated MAN measurements from ships shows that MODIS is meeting the pre-launch uncertainty estimates for aerosol optical depth (AOD) with 64% and 67% of retrievals at 550 nm, and 74% and 78% of retrievals at 870 nm, falling within expected uncertainty for Terra and Aqua, respectively. Angstrom Exponent comparisons show a high correlation between MODIS retrievals and shipboard measurements (R= 0.85 Terra, 0.83 Aqua), although the MODIS aerosol algorithm tends to underestimate particle size for large particles and overestimate size for small particles, as seen in earlier Collections. Prior analysis noted an offset between Terra and Aqua ocean AOD, without concluding which sensor was more accurate. The simple linear regression reported here, is consistent with other anecdotal evidence that Aqua agreement with AERONET is marginally better. However we cannot claim based on the current study that the better Aqua comparison is statistically significant. Systematic increase of error as a function of wind speed is noted in both Terra and Aqua retrievals. This wind speed dependency enters the retrieval when winds deviate from the 6 m/s value assumed in the rough ocean surface and white cap parameterizations. Wind speed dependency in the results can be mitigated by using auxiliary NCEP wind speed information in the retrieval process.

Kleidman, Richard G.↗
NASA NTRS2004

Accuracy Assessments and Validation of an Expanded UV Irradiance Database from Satellite Total Ozone Mapping Spectrometer (TOMS)

The TOMS UV irradiance database (1978 to 2000) has been expanded to include 5 new products (noon irradiance at 305, 310, 324, and 380 nm, and noon erythemal-weighted irradiance), in addition to the existing erythemal daily exposure, which permit direct Comparisons with ground-based measurements from UV spectrometers. Sensitivity studies are conducted to estimate uncertainties of the new TOMS UV irradiance data due to algorithm apriori assumptions. Comparisons with Brewer spectrometers as well as filter radiometers are used to review of the sources of known errors. Inability to distinguish between snow and cloud cover using only TOMS data results in large errors in estimating surface UV using snow climatology. A correction is suggested for the case when the regional snow albedo is known from an independent source. The summer-time positive bias between TOMS UV estimations and Brewer measurements can be seen at all wavelengths. This suggests the difference is not related to ozone absorption effects. We emphasize that uncertainty of boundary layer UV aerosol absorption properties remains a major source of error in modeling UV irradiance in clear sky conditions. Neglecting aerosol absorption by the present TOMS algorithm results in a positive summertime bias in clear-sky UV estimations over many locations. Due to high aerosol variability the bias is strongly site dependent. Data from UV-shadow-band radiometer and well-calibrated CIMEL sun-sky radiometer are used to quantify the bias at NASA/GSFC site in Greenbelt, MD. Recommendations are given to enable potential users to better account for local conditions by combining standard TOMS UV data with ancillary ground measurements.

Krotkov, N. A.↗