Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Statistical error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

SU-G-JeP1-12: Head-To-Head Performance Characterization of Two Multileaf Collimator Tracking Algorithms for Radiotherapy

Purpose: Multi-leaf collimator (MLC) tracking is being clinically pioneered to continuously compensate for thoracic and abdominal motion during radiotherapy. The purpose of this work is to characterize the performance of two MLC tracking algorithms for cancer radiotherapy, based on a direct optimization and a piecewise leaf fitting approach respectively. Methods: To test the algorithms, both physical and in silico experiments were performed. Previously published high and low modulation VMAT plans for lung and prostate cancer cases were used along with eight patient-measured organ-specific trajectories. For both MLC tracking algorithm, the plans were run with their corresponding patient trajectories. The physical experiments were performed on a Trilogy Varian linac and a programmable phantom (HexaMotion platform). For each MLC tracking algorithm, plan and patient trajectory, the tracking accuracy was quantified as the difference in aperture area between ideal and fitted MLC. To compare algorithms, the average cumulative tracking error area for each experiment was calculated. The two-sample Kolmogorov-Smirnov (KS) test was used to evaluate the cumulative tracking errors between algorithms. Results: Comparison of tracking errors for the physical and in silico experiments showed minor differences between the two algorithms. The KS D-statistics for the physical experiments were below 0.05 denoting no significant differences between the two distributions pattern and the average error area (direct optimization/piecewise leaf-fitting) were comparable (66.64 cm2/65.65 cm2). For the in silico experiments, the KS D-statistics were below 0.05 and the average errors area were also equivalent (49.38 cm2/48.98 cm2). Conclusion: The comparison between the two leaf fittings algorithms demonstrated no significant differences in tracking errors, neither in a clinically realistic environment nor in silico. The similarities in the two independent algorithms give confidence in the use of either algorithm for clinical implementation.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1973

Mariner Venus/Mercury 1973 navigation strategy

This paper presents the navigational aspects of the Mariner Venus/Mercury 1973 mission. Principal emphasis is on the development of the trajectory correction strategy, propellant costs, and delivery accuracies at Venus and Mercury. Key error sources and mission constraints are discussed. Of particular interest are the statistics of the first, post-Venus, maneuver which must correct for the magnification of errors in the Venus encounter. Finally, although not a primary objective of the mission, the analysis is extended to include a second Mercury encounter.

Mckinley, E. L.↗
NASA NTRS1996

Quantitative Measures for Software Independent Verification and Validation

As software is maintained or reused, it undergoes an evolution which tends to increase the overall complexity of the code. To understand the effects of this, we brought in statistics experts and leading researchers in software complexity, reliability, and their interrelationships. These experts' project has resulted in our ability to statistically correlate specific code complexity attributes, in orthogonal domains, to errors found over time in the HAL/S flight software which flies in the Space Shuttle. Although only a prototype-tools experiment, the result of this research appears to be extendable to all other NASA software, given appropriate data similar to that logged for the Shuttle onboard software. Our research has demonstrated that a more complete domain coverage can be mathematically demonstrated with the approach we have applied, thereby ensuring full insight into the cause-and-effects relationship between the complexity of a software system and the fault density of that system. By applying the operational profile we can characterize the dynamic effects of software path complexity under this same approach We now have the ability to measure specific attributes which have been statistically demonstrated to correlate to increased error probability, and to know which actions to take, for each complexity domain. Shuttle software verifiers can now monitor the changes in the software complexity, assess the added or decreased risk of software faults in modified code, and determine necessary corrections. The reports, tool documentation, user's guides, and new approach that have resulted from this research effort represent advances in the state of the art of software quality and reliability assurance. Details describing how to apply this technique to other NASA code are contained in this document.

Lee, Alice↗
NASA NTRS1993

The sensitivity of identified modal parameters to sensor placement errors and construction tolerances

This paper examines the sensitivity of experimentally measured modal frequencies and mode shapes to structural reassembly and sensor placement errors on a suspended three-bay truss. The statistical variations of identified mode shapes and frequencies are measured by repeated experiments. Both parameters are shown to vary measurably more with reassembly than sensor placement errors. Also, a directional stiffness in the truss joints was found to cause a parameter dependence on member orientation during reconstruction.

Hinkle, Jason↗
NASA NTRS2019

Multilevel Monte Carlo Estimation of Unbiased Expectation via Sample Reuse and the Low Variance Estimation of Asymptotic Rates

A new variant of the multilevel Monte Carlo estimator [5, 3, 9, 12] is presented for the estimation of expectation statistics that utilizes sample reuse in specified levels, explicitly removes approximation error bias associated with numerically computed output quantities of interest that have an asymptotic limit behavior, and permits a low variance estimate of the asymptotic rate of convergence to that limit. In addition, it is shown that this new multilevel Monte Carlo variant can yield a computational cost savings. A review of Monte Carlo and multilevel Monte Carlo estimators is presented that includes analysis of expected value, expected mean squared error, and the calculation of optimized multilevel sample size parameters. The multilevel Monte Carlo estimator produces estimates of expectation for numerically approximated output quantities of interest that are biased by approximation error. When the quantity of interest can be modeled as the asymptotic limit of numerically approximated output quantities of interest, it is theoretically possible to remove this approximation error bias in the multilevel Monte Carlo estimator. In actual implementations, however, this procedure is unreliable due to statistical variability and inaccuracy in estimating the needed asymptotic limit. Analysis and numerical experiment show that the proposed variant of the multilevel Monte Carlo method greatly reduces (in some cases eliminates) the statistical variability in this limit estimation.

Barth, Timothy↗
DOE OSTI2022

The impact of detection rate changes and correlations on random-coincidence background measurements

Coincidence detection of multiple particles emitted during an experiment can yield a new depth of understanding of the underlying process under study. However, the probability of detecting particles that are generated from the same physical event within a given coincidence time window is generally much lower than that of detecting particles that appear in the same coincidence time window, but were not created from the same physical event, and are therefore detected randomly in coincidence with each other. Thus, accurate and precise methods of measuring this random-coincidence background are essential for a wide variety of fields of science. A method to determine this background directly using the data themselves without any additional experimental run time or fake signals introduced in the data was recently established (O’Donnell, 2016). This method yields a statistical uncertainty on the random-coincidence background that is orders of magnitude smaller than that of the true coincidence data, though the potential for systematic errors of backgrounds from this method was never explored. In this work, we discuss common varieties of correlated and uncorrelated changes in the detection rates of each particle detected in an experiment. Here we demonstrate here that a correlation between particle detection rates from, for example, an incident particle beam that initiates a physical process of interest, creates systematic errors in the random-coincidence background measurement. We also discuss the impact of a variety of other realistic scenarios for rate changes in experiments. Lastly, a method is introduced to correct for errors in the random-coincidence background from any source, yielding an optimization between statistical precision and eliminating potential lingering systematic errors.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2015

Adaptive Bayesian Inference for Prediction

Performing rigorous predictive simulation necessitates that all available data be incorporated into the modeling process in order to accurately quantify uncertainties in predictions of the model. Statistical Bayesian inference is the most commonly used approach for incorporating data into uncertainty estimates. Statistical Bayesian inference uses data and an assumed error model to inform posterior distributions of model input variables and model discrepancies. An explicit characterization of the posterior distribution is not formed, but rather methods such as Markov chain Monte-Carlo (MCMC) are used to draw samples from the posterior. These samples are then used to evaluate statistics, such as integrals, of predicted quantities of interest (QoI). Recently the authors have developed an alternative reformulation of Bayesian inference based upon a measure-theoretic inversion technique. Unlike traditional statistical Bayesian inference, this reformulation allows us to directly interrogate the posterior distribution and, consequently, to generate samples from the posterior without MCMC. In this work we use forward propagation uncertainty quantification techniques to reduce the cost of approximating the posterior distribution. Specifically we investigate goal-oriented adaptive sparse grid approximation methods. These adaptive methods concentrate high-fidelity model evaluations in regions of the parameter space that both significantly contribute to the uncertainty in specified QoI and are informed by the available data. Numerical results are presented to compare our approach with the standard statistical Bayesian approach and to demonstrate the effectiveness of the goal-oriented adaptivity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Cosmic Voids in the SDSS DR12 BOSS Galaxy Sample: the Alcock–Paczyński test

We apply the Alcock–Paczyński (AP) test to stacked voids identified using the final data release (DR12) of the Baryon Oscillation Spectroscopic Survey (BOSS). We also use 1000 mock galaxy catalogs that match the geometry, density, and clustering properties of the BOSS sample in order to characterize the statistical uncertainties of our measurements and take into account systematic errors such as redshift space distortions. For both BOSS data and mock catalogs, we use the ZOBOV algorithm to identify voids, we stack together all voids with effective radii of 30--100 h{sup −1} Mpc in the redshift range of 0.43–0.7, and we accurately measure the shape of the stacked voids. Our tests with the mock catalogs show that we measure the stacked void ellipticity with a statistical precision of 2.6%. The stacked voids in redshift space are slightly squashed along the line of sight, consistent with previous studies. We repeat this measurement of stacked void shape in the BOSS data, assuming several values of Ω{sub m} within the flat ΛCDM model, and we compare this to the mock catalogs in redshift space to perform the AP test. We obtain a constraint of Ω{sub m}=0.38{sub −0.15}{sup +0.18} at the 68% confidence level from the AP test. We discuss the sources of statistical and systematic noise that affect the constraining power of this method. In particular, we find that the measured ellipticity of stacked voids changes more weakly with cosmology than the standard AP prediction, leading to significantly weaker constraints. We discuss how constraints will improve in future surveys with larger volumes and densities.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

ESM data downscaling: a comparison of super-resolution deep learning models

Abstract Climate projections at fine spatial resolutions are required to conduct accurate risk assessment for critical infrastructure and design adaptation planning. Generating these projections using advanced Earth system models (ESM) requires significant computational resources. To address this issue, various statistical downscaling techniques have been introduced to generate fine-resolution data from coarse-resolution simulations. In this study, we evaluate and compare five deep learning-based downscaling techniques, namely, super-resolution convolutional neural networks, fast super-resolution convolutional neural network ESM, efficient sub-pixel convolutional neural network, enhanced deep residual network (EDRN), and super-resolution generative adversarial network (SRGAN). These techniques are applied to a dataset generated by the Energy Exascale Earth System Model (E3SM), focusing on key surface variables such as surface temperature, shortwave heat flux, and longwave heat flux. Models are trained and validated using paired fine-resolution (0.25 $$^{\circ }$$ ∘ ) and coarse-resolution (1 $$^{\circ }$$ ∘ ) monthly data obtained from a 9-year simulation. Next, blind testing is performed using monthly data obtained from two different years outside of the training and validation set. To evaluate the efficiency of each technique, different statistical metrics are used, including mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The results show that EDRN outperforms other algorithms in terms of PSNR, SSIM, and MSE, but struggles to capture fine-scale features in the data. In contrast, SRGAN, a generative model that uses perceptual loss, excels in capturing fine details at boundaries and internal structures, resulting in lower LPIPS than other methods.

Pawar, Nikhil M. (ORCID:0000000211613289)↗
DOE OSTI2019

On the Effect of Sphere-Overlap on Super Coarse-Grained Models of Protein Assemblies

Ion mobility mass spectrometry (IM/MS) can provide structural information on intact protein complexes. Such data, including connectivity and collision cross sections (CCS) of assemblies’ subunits, can in turn be used as a guide to produce representative super coarse-grained models. These models are constituted by ensembles of overlapping spheres, each representing a protein subunit. A model is considered plausible if the CCS and sphere-overlap levels of its subunits fall within predetermined confidence intervals. While the first is determined by experimental error, the latter is based on a statistical analysis on a range of protein dimers. Here, we first propose a new expression to describe the overlap between two spheres. Then we analyze the effect of specific overlap cutoff choices on the precision and accuracy of super coarse-grained models. Finally, we propose a method to determine overlap cutoff levels on a per-case scenario, based on collected CCS data, and show that it can be applied to the characterization of the assembly topology of symmetrical homo-multimers. .

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2024

A novel conditional generative model for efficient ensemble forecasts of state variables in large-scale geological carbon storage

Integrating monitoring data to efficiently update reservoir pressure and CO 2 plume distribution forecasts presents a significant challenge in geological carbon storage (GCS) applications. Inverse modeling techniques are commonly used to fuse observational data and refine reservoir model parameters, thereby improving state variable forecasts. However, these techniques often rely on linear or Gaussian assumptions, which can limit their effectiveness in accurately predicting state variables. Moreover, simulating large-scale three-dimensional (3D) GCS problems is computationally expensive, making iterative runs in inverse problems prohibitive. To address these challenges, we propose a conditional generative model utilizing the score-based diffusion method for real-time 3D pressure and saturation field distribution predictions. Our approach involves solving the score function with a mini-batch-based Monte Carlo estimator to generate labeled data. This data is subsequently employed to train a fully connected neural network, enabling it to learn the conditional sample generator within a supervised learning framework. This method enables the rapid generation of a large ensemble of predictions, facilitating comprehensive uncertainty quantification of state variables. Here we applied our method to forecast the dynamic 3D distributions of pressure and saturation fields over a 30-year injection period. The statistical assessment with low root mean square error (RMSE) values demonstrates that our method can accurately predict the spatiotemporal distributions of both pressure and saturation fields. Moreover, the developed conditional generative model shows high computational efficiency by generating 100 ensemble forecasts of 3D state variables in less than 10 min. The consistency between ensemble averages and ground truth values further illustrates the model’s capability to capture state variable dynamics during the CO 2 plume injection process. Notably, the ground truth values fall within the ensemble forecasts, indicating that our uncertainty quantification effectively captures variability and potential noise in the observations. Thus, the developed conditional generative model proves to be a more efficient, accurate, and practical tool for GCS applications, facilitating timely risk analysis and informed decision-making.

58 GEOSCIENCES↗
DOE OSTI2018

Secondary carbides in high chromium cast irons: An alternative approach to their morphological and spatial distribution characterization

Highlights: • Systematic characterization of second phase particles by combining a variety of techniques • Comparison of morphological parameters obtained from SEM, BSE, etching+CLSM and EBSD • Quantitative analysis and degree of distribution of secondary carbides • Particle size, distribution and morphology from image analysis of CLSM, SEM and EBS images - Abstract: Secondary carbides precipitated in high chromium cast irons during thermal treatments were characterized by means of different characterization techniques, including scanning electron microscopy, energy dispersive X-ray spectroscopy, electron backscattered diffraction and a combination of chemical etching with confocal scanning laser microscopy. This set of techniques provides a full morphological, chemical and crystallographic description of the analysed phases. This work evaluated different methods for optimizing the image acquisition for a further image analysis (IA) based on the threshold binarization. Finally, the carbide size, distribution and morphology were determined after IA of the images acquired by aforementioned characterization techniques. Although the different techniques report some dispersion in the value for the average particle size, the particle inter-spacing and aspect ratio meet within the error value. The proposed characterization methodology provides statistically reliable data for a further evaluation of related physical properties in composites.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Automated Quantitative Analysis of Extended Irradiation Defects - Dislocations, Voids and Precipitates in Neutron Irradiated HT-9 Steel

Structural alloys that make up the core components of nuclear reactors are exposed to radiation that can substantially alter the microstructure and mechanical properties of the material. Irradiation can cause the formation of extended defects including dislocation line and loops, voids and various types of precipitates. To understand the irradiation effects, it is critical to characterize these defects individually and collectively to quantify how they evolve throughout irradiation stages. Transmission electron microscopy (TEM) is a standard tool for providing extensive size, number density and spatial distribution of extended irradiation defects, however, manual measurement of defect size and number density requires time-intensive effort to analyze enough defects that provide a statistically meaningful result and are prone to errors of subjectivity. In this work, we developed a set of MATLAB algorithms offering robust and automated identification and quantification of extended irradiation defects like dislocation lines, voids and precipitates commonly observed in neutron irradiated cladding alloys [1]. Using a neutron-irradiated HT-9 martensitic alloy irradiated at 412°C to 111.8 dpa, we demonstrate the capability of these algorithms for quantitative defect analysis. Figure 1 presents defect images of the HT-9 using an advanced diffraction contrast imaging scanning transmission electron microscopy (DCI STEM) that suppresses bend contours and provides better defect contrast compared to conventional TEM [2]. The first step of defect analysis is defect identification, a process that can be digitally achieved by contrast thresholding (for simple defect contrast) and by machine learning computer vision for defect semantic segmentation [3]. Here, labeled binary defect images (Figures 2a and 2c) were used. For dislocation density quantification, we employed an optimized circular-grid intersection method first introduced by Dr. John Steeds [4]. This method is shown in Figures 2a and 2b, where the number of intersections of dislocation lines with circular traces was measured. This produces an unbiased sampling of dislocation lines, regardless of their orientation. This process was optimized with automated digital intersection number and grid length registration. Interestingly, it was found that a relatively large number of circles in a set circular gird is was needed to produce an invariant result (Figure 2b), with a low systematic measurement error. This demonstrates the main advantage of computer-based defect analysis, in performing repetitive operations quickly with a reduced systematic error. For the quantitative analysis of voids and precipitates a challenge encountered with automated defect analysis is feature overlap, particularly when the density of voids/precipitates is high. To mitigate this problem, as shown in Figure 2d, we modified the popular circular Hough transformation and treated ‘circular shape’ as the particle descriptor for voids/precipitates, efficiently resolving the overlapping particles. In summary, this automated digital quantification together with machine learning enabled defect identification [3] opens new opportunities for standardizing the characterization workflow for quantitative analysis of extended irradiation defects.

Sainju, Rajat↗
DOE OSTI2023

Earth Mover’s Distance as a Metric to Evaluate the Extent of Charge Transfer in Excitations Using Discretized Real-Space Densities

This paper presents a novel theoretical measure, μ EMD , based on the earth mover's distance (EMD), for quantifying the density shift caused by electronic excitations in molecules. As input, the EMD metric uses only the discretized ground- and excited-state electron densities in real space, rendering it compatible with almost all electronic structure methods used to calculate excited states. The EMD metric is compared against other popular theoretical metrics for describing the extent of electron-hole separation in a wide range of excited states (valence, Rydberg, charge transfer, etc.). Further, the results showcase the EMD metric's effectiveness across all excitation types and suggest that it is useful as an additional tool to characterize electronic excitations. The study also reveals that μ EMD can function as a promising diagnostic tool for predicting the failure of pure exchange-correlation functionals. Specifically, we show statistical relationships among the functional-driven errors, the exact exchange content within the functional, and the magnitude of μ EMD values.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Quantifying the Variation in the Number of Donors in Quantum Dots Created Using Atomic Precision Advanced Manufacturing

Atomic-precision advanced manufacturing enables unique silicon quantum electronics built on quantum dots fabricated from small numbers of phosphorus dopants. The number of dopant atoms comprising a dot plays a central role in determining the behavior of charge and spin confined to the dots and thus overall device performance. Here in this work, we use both theoretical and experimental techniques to explore the combined impact of lithographic variation and stochastic kinetics on the number of P incorporations in quantum dots made using these techniques and how this variation changes as a function of the size of the dot. Using a kinetic model of PH3 dissociation augmented with novel reaction barriers, we demonstrate that for a 2 × 3 silicon dimer window the probability that no donor incorporates goes to zero, allowing for certainty in the placement of at least one donor. However, this still comes with some uncertainty in the precise number of incorporated donors (either one or two), and this variability may still impact certain applications. We also examine the impact of the size of the initial lithographic window, finding that the incorporation fraction saturates to δ-layer-like coverage as the circumference-to-area ratio decreases. We predict that this incorporation fraction depends strongly on the dosage of the precursor and that the standard deviation of the number of incorporations scales as ~√n, as would be expected for a sequence of largely independent incorporation events. Finally, we characterize an array of 36 experimentally prepared multidonor 3 × 3 nm lithographic windows with scanning tunneling microscopy, measuring the fidelity of the lithography to the desired array and the final location of PH x fragments within these lithographic windows. We use our kinetic model to examine the expected variability due to the observed lithographic error, predicting a negligible impact on incorporation statistics. We find good agreement between our model and the inferred incorporation locations in these windows from scanning tunneling microscope measurements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Regularized machine learning on molecular graph model explains systematic error in DFT enthalpies

Abstract A major goal of materials research is the discovery of novel and efficient heterogeneous catalysts for various chemical processes. In such studies, the candidate catalyst material is modeled using tens to thousands of chemical species and elementary reactions. Density Functional Theory (DFT) is widely used to calculate the thermochemistry of these species which might be surface species or gas-phase molecules. The use of an approximate exchange correlation functional in the DFT framework introduces an important source of error in such models. This is especially true in the calculation of gas phase molecules whose thermochemistry is calculated using the same planewave basis set as the rest of the surface mechanism. Unfortunately, the nature and magnitude of these errors is unknown for most practical molecules. Here, we investigate the error in the enthalpy of formation for 1676 gaseous species using two different DFT levels of theory and the ‘ground truth values’ obtained from the NIST database. We featurize molecules using graph theory. We use a regularized algorithm to discover a sparse model of the error and identify important molecular fragments that drive this error. The model is robust to rigorous statistical tests and is used to correct DFT thermochemistry, achieving more than an order of magnitude improvement.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Measurement of Telescope Transmission Using a Collimated Beam Projector

With the increasingly large number of Type Ia supernova being detected by current-generation survey telescopes, and even more expected with the upcoming Rubin Observatory Legacy Survey of Space and Time, the precision of cosmological measurements will become limited by systematic uncertainties in flux calibration rather than statistical noise. One major source of systematic error in determining SNe Ia color evolution (needed for distance estimation) is uncertainty in telescope transmission, both within and between surveys. We introduce here the Collimated Beam Projector (CBP), which is meant to measure a telescope transmission with collimated light. The collimated beam more closely mimics a stellar wave front as compared to flat-field-based instruments, allowing for more precise handling of systematic errors such as those from ghosting and filter angle-of-incidence dependence. As a proof of concept, we present CBP measurements of the StarDICE prototype telescope, achieving a standard (1σ) uncertainty of 3% on average over the full wavelength range measured with a single beam illumination.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2013

Prediction of Regulation Reserve Requirements in California ISO Control Area based on BAAL Standard

This paper presents new methodologies developed at Pacific Northwest National Laboratory (PNNL) to estimate regulation capacity requirements in the California ISO control area. Two approaches have been developed: (1) an approach based on statistical analysis of actual historical area control error (ACE) and regulation data, and (2) an approach based on balancing authority ACE limit control performance standard. The approaches predict regulation reserve requirements on a day-ahead basis including upward and downward requirements, for each operating hour of a day. California ISO data has been used to test the performance of the proposed algorithms. Results show that software tool allows saving up to 30% on the regulation procurements cost .

Etingov, Pavel V.↗