Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Statistical error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

SAD phasing of XFEL data depends critically on the error model

A nonlinear least-squares method for refining a parametric expression describing the estimated errors of reflection intensities in serial crystallographic (SX) data is presented. This approach, which is similar to that used in the rotation method of crystallographic data collection at synchrotrons, propagates error estimates from photon-counting statistics to the merged data. Here, it is demonstrated that the application of this approach to SX data provides better SAD phasing ability, enabling the autobuilding of a protein structure that had previously failed to be built. Estimating the error in the merged reflection intensities requires the understanding and propagation of all of the sources of error arising from the measurements. One type of error, which is well understood, is the counting error introduced when the detector counts X-ray photons. Thus, if other types of random errors (such as readout noise) as well as uncertainties in systematic corrections (such as from X-ray attenuation) are completely understood, they can be propagated along with the counting error, as appropriate. In practice, most software packages propagate as much error as they know how to model and then include error-adjustment terms that scale the error estimates until they explain the variance among the measurements. If this is performed carefully, then during SAD phasing likelihood-based approaches can make optimal use of these error estimates, increasing the chance of a successful structure solution. In serial crystallography, SAD phasing has remained challenging, with the few examples of de novo protein structure solution each requiring many thousands of diffraction patterns. Here, the effects of different methods of treating the error estimates are estimated and it is shown that using a parametric approach that includes terms proportional to the known experimental uncertainty, the reflection intensity and the squared reflection intensity to improve the error estimates can allow SAD phasing even from weak zinc anomalous signal.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2014

Guide to Using the WIND Toolkit Validation Code

In response to the U.S. Department of Energy's goal of using 20% wind energy by 2030, the Wind Integration National Dataset (WIND) Toolkit was created to provide information on wind speed, wind direction, temperature, surface air pressure, and air density on more than 126,000 locations across the United States from 2007 to 2013. The numerical weather prediction model output, gridded at 2-km and at a 5-minute resolution, was further converted to detail the wind power production time series of existing and potential wind facility sites. For users of the dataset it is important that the information presented in the WIND Toolkit is accurate and that errors are known, as then corrective steps can be taken. Therefore, we provide validation code written in R that will be made public to provide users with tools to validate data of their own locations. Validation is based on statistical analyses of wind speed, using error metrics such as bias, root-mean-square error, centered root-mean-square error, mean absolute error, and percent error. Plots of diurnal cycles, annual cycles, wind roses, histograms of wind speed, and quantile-quantile plots are created to visualize how well observational data compares to model data. Ideally, validation will confirm beneficial locations to utilize wind energy and encourage regional wind integration studies using the WIND Toolkit.

17 WIND ENERGY↗
DOE OSTI2017

Validation of sea ice models using an uncertainty-based distance metric for multiple model variables: NEW METRIC FOR SEA ICE MODEL VALIDATION

Here, we implement a variance-based distance metric (D n ) to objectively assess skill of sea ice models when multiple output variables or uncertainties in both model predictions and observations need to be considered. The metric compares observations and model data pairs on common spatial and temporal grids improving upon highly aggregated metrics (e.g., total sea ice extent or volume) by capturing the spatial character of model skill. The D n metric is a gamma-distributed statistic that is more general than the χ 2 statistic commonly used to assess model fit, which requires the assumption that the model is unbiased and can only incorporate observational error in the analysis. The D n statistic does not assume that the model is unbiased, and allows the incorporation of multiple observational data sets for the same variable and simultaneously for different variables, along with different types of variances that can characterize uncertainties in both observations and the model. This approach represents a step to establish a systematic framework for probabilistic validation of sea ice models. The methodology is also useful for model tuning by using the D n metric as a cost function and incorporating model parametric uncertainty as part of a scheme to optimize model functionality. We apply this approach to evaluate different configurations of the standalone Los Alamos sea ice model (CICE) encompassing the parametric uncertainty in the model, and to find new sets of model configurations that produce better agreement than previous configurations between model and observational estimates of sea ice concentration and thickness.

58 GEOSCIENCES↗
DOE OSTI2015

IMFIT: A FAST, FLEXIBLE NEW PROGRAM FOR ASTRONOMICAL IMAGE FITTING

I describe a new, open-source astronomical image-fitting program called IMFIT, specialized for galaxies but potentially useful for other sources, which is fast, flexible, and highly extensible. A key characteristic of the program is an object-oriented design that allows new types of image components (two-dimensional surface-brightness functions) to be easily written and added to the program. Image functions provided with IMFIT include the usual suspects for galaxy decompositions (Sérsic, exponential, Gaussian), along with Core-Sérsic and broken-exponential profiles, elliptical rings, and three components that perform line-of-sight integration through three-dimensional luminosity-density models of disks and rings seen at arbitrary inclinations. Available minimization algorithms include Levenberg-Marquardt, Nelder-Mead simplex, and Differential Evolution, allowing trade-offs between speed and decreased sensitivity to local minima in the fit landscape. Minimization can be done using the standard χ{sup 2} statistic (using either data or model values to estimate per-pixel Gaussian errors, or else user-supplied error images) or Poisson-based maximum-likelihood statistics; the latter approach is particularly appropriate for cases of Poisson data in the low-count regime. I show that fitting low-signal-to-noise ratio galaxy images using χ{sup 2} minimization and individual-pixel Gaussian uncertainties can lead to significant biases in fitted parameter values, which are avoided if a Poisson-based statistic is used; this is true even when Gaussian read noise is present.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Correction of Dual-PRF Doppler Velocity Outliers in the Presence of Aliasing

Abstract In Doppler weather radars, the presence of unfolding errors or outliers is a well-known quality issue for radial velocity fields estimated using the dual–pulse repetition frequency (PRF) technique. Postprocessing methods have been developed to correct dual-PRF outliers, but these need prior application of a dealiasing algorithm for an adequate correction. This paper presents an alternative procedure based on circular statistics that corrects dual-PRF errors in the presence of extended Nyquist aliasing. The correction potential of the proposed method is quantitatively tested by means of velocity field simulations and is exemplified in the application to real cases, including severe storm events. The comparison with two other existing correction methods indicates an improved performance in the correction of clustered outliers. The technique proposed is well suited for real-time applications requiring high-quality Doppler radar velocity fields, such as wind shear and mesocyclone detection algorithms, or assimilation in numerical weather prediction models.

58 GEOSCIENCES↗
DOE OSTI2015

Characterizing Quantum Devices Using Model Selection

Quantum information processors are a new kind of computing device with the potential to solve certain problems very quickly. The quantum bits, or "ʺqubits,"ʺ that enable such processors are being developed at Sandia and around the world. In support of this effort, Sandia’s Center for Computing Research is designing methods and algorithms to characterize and debug the behavior of experimental qubits, such as trapped ions and electron spins in silicon. This is a challenge, because qubits are hard to characterize for exactly the same reason that they are powerful. Their behavior (and misbehavior) is far richer and more complicated than that of classical bits! The most reliable models of quantum errors have infinitely many parameters, and we never have enough data to fit all of them. Until now, researchers simply threw out most of these parameters resulting in a crude approximation that makes the problem tractable, but isn’t reliable or accurate enough for Sandia’s experimental work. Our new techniques use statistical model selection to identify “significant” error parameters on the fly, applying a data-­‐‑driven information criterion to choose a Hilbert space that describes the data well without overfitting. To find a reliable criterion, we needed to evaluate many candidates on an enormous corpus of test cases–for which we relied on Red Sky’s ability to run parallel Monte Carlo simulations. Our computations demonstrate how classic model selection methods have to be modified for quantum devices. These results will enable new techniques that identify the effective dimension of as-­‐‑built qubits on the fly, and then use this information to reliably pin down error behaviors and produce successively better and more useful qubits. This project highlights the close and productive relationship between Sandia and the University of New Mexico, where Travis Scholten is working towards a Ph.D. in physics. Both the intellectual environment of UNM and the high-­‐‑powered supercomputing available at Sandia were critical ingredients in achieving our research goals.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Tunable power law in the desynchronization events of coupled chaotic electronic circuits

We study the statistics of the amplitude of the synchronization error in chaotic electronic circuits coupled through linear feedback. Depending on the coupling strength, our system exhibits three qualitatively different regimes of synchronization: weak coupling yields independent oscillations; moderate to strong coupling produces a regime of intermittent synchronization known as attractor bubbling; and stronger coupling produces complete synchronization. In the regime of moderate coupling, the probability distribution for the sizes of desynchronization events follows a power law, with an exponent that can be adjusted by changing the coupling strength. Such power-law distributions are interesting, as they appear in many complex systems. However, most of the systems with such a behavior have a fixed value for the exponent of the power law, while here we present an example of a system where the exponent of the power law is easily tuned in real time.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

Full-stack Quantification of Variability in Predicting Ion Transport Properties using Machine-learned Interatomic Potentials

Machine-learned interatomic potentials (MLIPs) have become the state-of-the-art for performing accurate, scalable molecular dynamics (MD) simulations. It is therefore crucial to understand and quantify the reliability of MLIPs for downstream property predictions. Uncertainty in predicted properties can arise from limitations in first-principles training data, intrinsic MLIP model errors in representing the data, and the statistical noise introduced during subsequent MD simulations. Using ion transport in Li7P3S11 as a case study, we systematically assess the impact of training set size and selection, neural network stochasticity, and MD sampling statistics on predicted diffusivity and activation energy. We find that when using equivariant MLIP architectures with standard MD protocols, uncertainty arising from MD sampling dominates over model-induced errors. In contrast, MLIP errors relative to the underlying first-principles data are consistently minor. Given this, there are two main routes to improving the accuracy of predictions based on MLIP potentials: adopting higher accuracy reference data generation methods, and improving the MD sampling statistics.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Modeling energy balance while correcting for measurement error via free knot splines

Measurements of energy balance components (energy intake, energy expenditure, changes in energy stores) are often plagued with measurement error. Doubly-labeled water can measure energy intake (EI) with negligible error, but is expensive and cumbersome. An alternative approach that is gaining popularity is to use the energy balance principle, by measuring energy expenditure (EE) and change in energy stores (ES) and then back-calculate EI. Gold standard methods for EE and ES exist and are known to give accurate measurements, albeit at a high cost. We propose a joint statistical model to assess the measurement error in cheaper, non-intrusive measures of EE and ES. We let the unknown true EE and ES for individuals be latent variables, and model them using a bivariate distribution. We try both a bivariate Normal as well as a Dirichlet Process Mixture Model, and compare the results via simulation. Our approach, is the first to account for the dependencies that exist in individuals’ daily EE and ES. We employ semiparametric regression with free knot splines for measurements with error, and linear components for error free covariates. We adopt a Bayesian approach to estimation and inference and use Reversible Jump Markov Chain Monte Carlo to generate draws from the posterior distribution. Based on the semiparameteric regression, we develop a calibration equation that adjusts a cheaper, less reliable estimate, closer to the true value. Along with this calibrated value, our method also gives credible intervals to assess uncertainty. A simulation study shows our calibration helps produce a more accurate estimate. Furthermore, our approach compares favorably in terms of prediction to other commonly used models.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

Quantitative validation of carbon-fiber laminate low velocity impact simulations

Simulations of low velocity impact with a flat cylindrical indenter upon a carbon fiber fabric reinforced polymer laminate are rigorously validated. Comparison of the impact energy absorption between the model and experiment is used as the validation metric. Additionally, non-destructive evaluation, including ultrasonic scans and three-dimensional computed tomography, provide qualitative validation of the models. The simulations include delamination, matrix cracks and fiber breaks. An orthotropic damage and failure constitutive model, capable of predicting progressive damage and failure, is developed in conjunction and described. An ensemble of simulations incorporating model parameter uncertainties is used to predict a response distribution which is then compared to experimental output using appropriate statistical methods. Lastly, the model form errors are exposed and corrected for use in an additional blind validation analysis. The result is a quantifiable confidence in material characterization and model physics when simulating low velocity impact in structures of interest.

36 MATERIALS SCIENCE↗
DOE OSTI2016

Robust verification analysis

We introduce a new methodology for inferring the accuracy of computational simulations through the practice of solution verification. We demonstrate this methodology on examples from computational heat transfer, fluid dynamics and radiation transport. Our methodology is suited to both well- and ill-behaved sequences of simulations. Our approach to the analysis of these sequences of simulations incorporates expert judgment into the process directly via a flexible optimization framework, and the application of robust statistics. The expert judgment is systematically applied as constraints to the analysis, and together with the robust statistics guards against over-emphasis on anomalous analysis results. We have named our methodology Robust Verification. Our methodology is based on utilizing multiple constrained optimization problems to solve the verification model in a manner that varies the analysis' underlying assumptions. Constraints applied in the analysis can include expert judgment regarding convergence rates (bounds and expectations) as well as bounding values for physical quantities (e.g., positivity of energy or density). This approach then produces a number of error models, which are then analyzed through robust statistical techniques (median instead of mean statistics). This provides self-contained, data and expert informed error estimation including uncertainties for both the solution itself and order of convergence. Our method produces high quality results for the well-behaved cases relatively consistent with existing practice. The methodology can also produce reliable results for ill-behaved circumstances predicated on appropriate expert judgment. We demonstrate the method and compare the results with standard approaches used for both code and solution verification on well-behaved and ill-behaved simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2019

Data clustering for the high-resolution alignment of microstructure and strain fields

The emergence of small-scale deformation mapping by a combination of scanning electron microscopy and digital image correlation (SEM-DIC) has enabled full-field investigations into the complex roles of microstructure on microscale deformation mechanisms. However, these investigations are hindered by errors in alignment between the microstructure data, generally acquired by electron backscatter diffraction (EBSD), and the microscale strain data obtained by SEM-DIC. Distortions, stitching artifacts, and spatial resolution differences between microstructure and strain fields can lead to misalignments on the order of μms. This alignment uncertainty introduces error into microstructure-strain localization analyses and precludes the examination of deformation mechanisms near grain boundaries. To improve alignment between EBSD-obtained grain boundaries and SEM-DIC strain data, an approach was created wherein a mantle was first established around the EBSD-acquired grain boundaries. Strain data was then clustered within this mantle using a k-means algorithm to identify grain boundary strain localization, and these boundary points were fit to obtain a continuous curve for each individual boundary. Clustered point outliers, such as those due to grain boundary-local dislocation slip, were statistically identified by using an absolute error threshold and removed from the grain boundary fit. The ending identification of grain boundaries in the microscale strain data is significantly improved from EBSD-identified boundaries.

36 MATERIALS SCIENCE↗
DOE OSTI2015

Notes on power of normality tests of error terms in regression models

Normality is one of the basic assumptions in applying statistical procedures. For example in linear regression most of the inferential procedures are based on the assumption of normality, i.e. the disturbance vector is assumed to be normally distributed. Failure to assess non-normality of the error terms may lead to incorrect results of usual statistical inference techniques such as t-test or F-test. Thus, error terms should be normally distributed in order to allow us to make exact inferences. As a consequence, normally distributed stochastic errors are necessary in order to make a not misleading inferences which explains a necessity and importance of robust tests of normality. Therefore, the aim of this contribution is to discuss normality testing of error terms in regression models. In this contribution, we introduce the general RT class of robust tests for normality, and present and discuss the trade-off between power and robustness of selected classical and robust normality tests of error terms in regression models.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

An evaluation of multi-fidelity methods for quantifying uncertainty in projections of ice-sheet mass change

Abstract. This study investigated the computational benefits of using multi-fidelity statistical estimation (MFSE) algorithms to quantify uncertainty in the mass change of Humboldt Glacier, Greenland, between 2007 and 2100 using a single climate change scenario. The goal of this study was to determine whether MFSE can use multiple models of varying cost and accuracy to reduce the computational cost of estimating the mean and variance of the projected mass change of a glacier. The problem size and complexity were chosen to reflect the challenges posed by future continental-scale studies while still facilitating a computationally feasible investigation of MFSE methods. When quantifying uncertainty introduced by a high-dimensional parameterization of the basal friction field, MFSE was able to reduce the mean-squared error in the estimates of the statistics by well over an order of magnitude when compared to a single-fidelity approach that only used the highest-fidelity model. This significant reduction in computational cost was achieved despite the low-fidelity models used being incapable of capturing the local features of the ice-flow fields predicted by the high-fidelity model. The MFSE algorithms were able to effectively leverage the high correlation between each model's predictions of mass change, which all responded similarly to perturbations in the model inputs. Consequently, our results suggest that MFSE could be highly useful for reducing the cost of computing continental-scale probabilistic projections of sea-level rise due to ice-sheet mass change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2015

SU-E-T-769: T-Test Based Prior Error Estimate and Stopping Criterion for Monte Carlo Dose Calculation in Proton Therapy

Purpose: The Monte Carlo (MC) method is a gold standard for dose calculation in radiotherapy. However, it is not a priori clear how many particles need to be simulated to achieve a given dose accuracy. Prior error estimate and stopping criterion are not well established for MC. This work aims to fill this gap. Methods: Due to the statistical nature of MC, our approach is based on one-sample t-test. We design the prior error estimate method based on the t-test, and then use this t-test based error estimate for developing a simulation stopping criterion. The three major components are as follows.First, the source particles are randomized in energy, space and angle, so that the dose deposition from a particle to the voxel is independent and identically distributed (i.i.d.).Second, a sample under consideration in the t-test is the mean value of dose deposition to the voxel by sufficiently large number of source particles. Then according to central limit theorem, the sample as the mean value of i.i.d. variables is normally distributed with the expectation equal to the true deposited dose.Third, the t-test is performed with the null hypothesis that the difference between sample expectation (the same as true deposited dose) and on-the-fly calculated mean sample dose from MC is larger than a given error threshold, in addition to which users have the freedom to specify confidence probability and region of interest in the t-test based stopping criterion. Results: The method is validated for proton dose calculation. The difference between the MC Result based on the t-test prior error estimate and the statistical Result by repeating numerous MC simulations is within 1%. Conclusion: The t-test based prior error estimate and stopping criterion are developed for MC and validated for proton dose calculation. Xiang Hong and Hao Gao were partially supported by the NSFC (#11405105), the 973 Program (#2015CB856000) and the Shanghai Pujiang Talent Program (#14PJ1404500)

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

FRAM Version 6.1’s Bias

The Fixed-Energy Response-Function Analysis with Multiple Efficiency (FRAM) code was developed at Los Alamos National Laboratory to measure the gamma-ray spectrometry of the isotopic composition of plutonium, uranium, and other actinides. For FRAM version 4 and earlier, the reported uncertainties of the results come from the propagation of the statistics in the peak areas only. No systematic error components are included in the reported uncertainties. We did several studies with FRAM v.4 and found that FRAM’s statistical precision can be reasonably represented by its reported uncertainties. FRAM’s biases or systematic uncertainties can come from a variety of sources and can be difficult to determine. For FRAM version 5, we carefully examined the FRAM analytical results of both the archival plutonium data and the data specifically acquired for the isotopic uncertainty analysis project and found the relationship between the bias and other parameters. We then worked out the equations representing the biases of the measured isotopes from each measurement using internal spectral parameters, such as peak resolution and shape, region of analysis, and burnup (for plutonium) or enrichment (for uranium). The resulting biases were included in the reported uncertainties of FRAM v.5. For the upcoming FRAM version 6.1, we are doing the same study that we did for FRAM v.5. The resulting biases will be included in its reported uncertainties.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Simulating Z 2 lattice gauge theory on a quantum computer

The utility of quantum computers for simulating lattice gauge theories is currently limited by the noisiness of the physical hardware. Various quantum error mitigation strategies exist to reduce the statistical and systematic uncertainties in quantum simulations via improved algorithms and analysis strategies. We perform quantum simulations of Z 2 gauge theory with matter to study the efficacy and interplay of different error mitigation methods: readout error mitigation, randomized compiling, rescaling, and dynamical decoupling. We compute Minkowski correlation functions in this confining gauge theory and extract the mass of the lightest spin-1 state from fits to their time dependence. Quantum error mitigation extends the range of times over which our correlation function calculations are accurate by a factor of 6 and is therefore essential for obtaining reliable masses.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗