Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “error analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

IMRT QA: Selecting gamma criteria based on error detection sensitivity

Purpose: The gamma comparison is widely used to evaluate the agreement between measurements and treatment planning system calculations in patient-specific intensity modulated radiation therapy (IMRT) quality assurance (QA). However, recent publications have raised concerns about the lack of sensitivity when employing commonly used gamma criteria. Understanding the actual sensitivity of a wide range of different gamma criteria may allow the definition of more meaningful gamma criteria and tolerance limits in IMRT QA. We present a method that allows the quantitative determination of gamma criteria sensitivity to induced errors which can be applied to any unique combination of device, delivery technique, and software utilized in a specific clinic. Methods: A total of 21 DMLC IMRT QA measurements (ArcCHECK®, Sun Nuclear) were compared to QA plan calculations with induced errors. Three scenarios were studied: MU errors, multi-leaf collimator (MLC) errors, and the sensitivity of the gamma comparison to changes in penumbra width. Gamma comparisons were performed between measurements and error-induced calculations using a wide range of gamma criteria, resulting in a total of over 20 000 gamma comparisons. Gamma passing rates for each error class and case were graphed against error magnitude to create error curves in order to represent the range of missed errors in routine IMRT QA using 36 different gamma criteria. Results: This study demonstrates that systematic errors and case-specific errors can be detected by the error curve analysis. Depending on the location of the error curve peak (e.g., not centered about zero), 3%/3 mm threshold = 10% at 90% pixels passing may miss errors as large as 15% MU errors and ±1 cm random MLC errors for some cases. As the dose threshold parameter was increased for a given %Diff/distance-to-agreement (DTA) setting, error sensitivity was increased by up to a factor of two for select cases. This increased sensitivity with increasing dose threshold was consistent across all studied combinations of %Diff/DTA. Criteria such as 2%/3 mm and 3%/2 mm with a 50% threshold at 90% pixels passing are shown to be more appropriately sensitive without being overly strict. However, a broadening of the penumbra by as much as 5 mm in the beam configuration was difficult to detect with commonly used criteria, as well as with the previously mentioned criteria utilizing a threshold of 50%. Conclusions: We have introduced the error curve method, an analysis technique which allows the quantitative determination of gamma criteria sensitivity to induced errors. The application of the error curve method using DMLC IMRT plans measured on the ArcCHECK® device demonstrated that large errors can potentially be missed in IMRT QA with commonly used gamma criteria (e.g., 3%/3 mm, threshold = 10%, 90% pixels passing). Additionally, increasing the dose threshold value can offer dramatic increases in error sensitivity. This approach may allow the selection of more meaningful gamma criteria for IMRT QA and is straightforward to apply to other combinations of devices and treatment techniques.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2022

Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence

Computational models of the Earth System are critical tools for modern scientific inquiry. Effortstoward evaluating and improving errors in representations of physical and chemical processes inthese large computational systems are commonly stymied by highly nonlinear and complexerror behavior. Recent work has shown that these errors can be effectively predicted usingmodern Artificial Intelligence (A.I.) techniques. In this work, we go beyond these previousstudies to apply an interpretable A.I. technique to not only predict model errors but also movetoward understanding the underlying reasons for successful error prediction. We use XGBoostclassification trees and SHapley Additive exPlanations (SHAP) analysis to explore the errors inthe prediction of lightning occurrence in the NASA GEOS model, a widely used Earth SystemModel. This explainable error prediction system can effectively predict the model error andindicates that the errors are strongly related to convective processes and the characteristics ofthe land surface.

Artificial intelligence↗
DOE OSTI2024

On the Representativity of Electrode Microstructure Parameters and Their Electrochemical Response for Lithium Ion Batteries

Lithium-ion battery electrochemical models require an accurate description of the electrodes microstructures to be predictive that can be achieved through nanoscale imaging. Such observations are however limited by their field of view (FOV), as they provide only a subset of the whole electrode volume that does not necessarily represent the whole electrode microstructure heterogeneity, and therefore can bias the microstructure analysis. A microstructure scale electrochemical model was used to investigate lithium plating onset, material non-uniform utilization, and in-plane heterogeneities for an NMC-graphite full cell. To evaluate the representativeness, and thus relevance, of these model predictions, a coupled representativity analysis has been performed on the microstructure parameters and, in a novel way, on the full cell electrochemical response. Electrode microstructure parameters representativeness has been first quantified using the representative volume element (RVE) methodology. The RVE major flaw is that ultimately it can only conclude if a FOV contains representative subvolumes of the FOV, but not if the FOV itself is representative of the electrode volume. Analysis can conclude negatively ('FOV is not representative'), but not positively ('FOV is representative'). One major contribution of this work was to quantify the convergence of the RVE size with the FOV, to actually investigate the FOV representativeness and thus partly remedy this intrinsic limitation. The analysis determined that performing a standard RVE calculation, without exploring its FOV convergence, is likely to strongly underestimate the actual RVE size. The new RVE methodology has been automated in the NREL open-source Microstructure Analysis Toolbox (MATBOX) and is available to the battery community. Representativeness of microstructure parameters is however only an intermediate step, as the end-results of an electrochemical model are performances predictions. Indeed, what is the practical consequence of a given deviation for a microstructure parameter? The microstructure parameter deviation propagations to the 3D microstructure scale electrochemical response have been then quantified for different charge rates. This defines a threshold for the microstructure parameters FOV for a desired maximum deviation of the electrochemical response. Such deviation propagation analysis is analogous to error propagation analysis and is necessary to determine the relevance of microstructure scale model predictions for macroscale predictions. Electrochemical model shows cell representative section areas are increasing with C-rate, due to higher in-plane heterogeneities, indicating larger FOVs are required specifically for fast charge modeling. Therefore, we introduced the novel concept of electrochemical RVE (eRVE) that is a function of the operating conditions (thus defined as a dynamic RVE), with an increasing dependence with the C-rate. Representativity analysis of the investigated cell determined a FOV of 144.4 x 54.4 m2 is large enough to establish a convergence on the representative section areas for low to intermediate C-rate (=2.5C), but not large enough to conclude for higher rates. This work aims to emphasize the importance of representativity analysis for LIB electrode microstructures, as it is required to estimate the error, and thus the relevance, of microstructure parameters intended to be used in macroscale models. The methodology and results can help researchers to select the relevant imaging and associated FOV required to provide accurate enough microstructure parameters.

ADVANCED PROPULSION SYSTEMS↗
NASA NTRS1990

Sensitivity of feedforward neural networks to weight errors

An analysis is made of the sensitivity of feedforward layered networks of Adaline elements (threshold logic units) to weight errors. An approximation is derived which expresses the probability of error for an output neuron of a large network (a network with many neurons per layer) as a function of the percentage change in the weights. As would be expected, the probability of error increases with the number of layers in the network and with the percentage change in the weights. The probability of error is essentially independent of the number of weights per neuron and of the number of neurons per layer, as long as these numbers are large (on the order of 100 or more).

Stevenson, Maryhelen↗
DOE OSTI2022

Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence

Abstract Computational models of the Earth System are critical tools for modern scientific inquiry. Efforts toward evaluating and improving errors in representations of physical and chemical processes in these large computational systems are commonly stymied by highly nonlinear and complex error behavior. Recent work has shown that these errors can be effectively predicted using modern Artificial Intelligence (A.I.) techniques. In this work, we go beyond these previous studies to apply an explainable A.I. technique to not only predict model errors but also move toward understanding the underlying reasons for successful error prediction. We use XGBoost classification trees and SHapley Additive exPlanations analysis to explore the errors in the prediction of lightning occurrence in the NASA Goddard Earth Observing System model, a widely used Earth System Model. This explainable error prediction system can effectively predict the model error and indicates that the errors are strongly related to convective processes and the characteristics of the land surface.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1979

Navigation and meteorological error equations for some aerodynamic parameters

Mathematical equations for the analysis of the errors that are expected in a set of postflight aerodynamic parameters are presented. The errors are due to inaccuracies in the Shuttle best estimate trajectory and in the meteorological data obtained in support of the flights. The error analysis shows that the parameter vector, Z, and its associated error covariance matrix, C sub Z, is calculated from a given state vector, X, and its associated covariance matrix, C sub X.

Krikorian, M. J.↗
NASA NTRS1991

Sources of error in planetary photoclinometry

A quantitative analysis of error sources in 1D planetary photoclinometry is presented. The technique is affected by error sources arising from the spacecraft image, the planetary body, and the scan line orientation. Slope errors are calculated for each of these sources, using examples of Voyager imaging of Ganymede and Viking orbiting imaging of Mars. Slope errors are investigated for a variety of viewing and lighting geometries, slope angles, and slope orientations. The results are broken down into nonsystematic and systematic errors. Derivations that allow the calculation of photoclinometric slope errors for any photometric function are presented, and the implications of these results for 2D photoclinometric techniques are discussed.

Jankowski, David G.↗
NASA NTRS1975

Wavefront-error evaluation by mathematical analysis of experimental Foucault-test data

The diffraction theory of the Foucault test provides an integral formula expressing the complex amplitude and irradiance distribution in the Foucault pattern of a test mirror (lens) as a function of wavefront error. Recent literature presents methods of inverting this formula to express wavefront error in terms of irradiance in the Foucault pattern. The present paper describes a study in which the inversion formulation was applied to photometric Foucault-test measurements on a nearly diffraction-limited mirror to determine wavefront errors for direct comparison with ones determined from scatter-plate interferometer measurements. The results affirm the practicability of the Foucault test for quantitative wavefront analysis of very small errors, and they reveal the fallacy of the prevalent belief that the test is limited to qualitative use only. Implications of the results with regard to optical testing and the potential use of the Foucault test for wavefront analysis in orbital space telescopes are discussed.

Wilson, R. G.↗
NASA NTRS1983

Skeletal and body composition evaluation

Research on radiation detectors for absorptiometry; analysis of errors affective single photon absorptiometry and development of instrumentation; analysis of errors affecting dual photon absorptiometry and development of instrumentation; comparison of skeletal measurements with other techniques; cooperation with NASA projects for skeletal evaluation in spaceflight (Experiment MO-78) and in laboratory studies with immobilized animals; studies of postmenopausal osteoporosis; organization of scientific meetings and workshops on absorptiometric measurement; and development of instrumentation for measurement of fluid shifts in the human body were performed. Instrumentation was developed that allows accurate and precise (2% error) measurements of mineral content in compact and trabecular bone and of the total skeleton. Instrumentation was also developed to measure fluid shifts in the extremities. Radiation exposure with those procedures is low (2-10 MREM). One hundred seventy three technical reports and one hundred and four published papers of studies from the University of Wisconsin Bone Mineral Lab are listed.

Mazess, R. B.↗
NASA NTRS1992

r-adaptive methods based on element-level error indicators for parallel analysis of plates and shells

The formulation and application of element-level error indicators based on parametrized variational principles are investigated. These indicators are ideally suited to drive adaptive mesh refinment on parallel computers where access to neighboring elements resident on different processors may incur significant computational overhead. Furthermore, such estimators are not affected by physical jumps at wavefronts or interfaces. An estimator derived from the higher-order element energy is applied to r adaptation of meshes in plates and shell structures where the exact solution is known.

Militello, Carmelo↗
NASA NTRS2011

Command Process Modeling & Risk Analysis

Commanding Errors may be caused by a variety of root causes. It's important to understand the relative significance of each of these causes for making institutional investment decisions. One of these causes is the lack of standardized processes and procedures for command and control. We mitigate this problem by building periodic tables and models corresponding to key functions within it. These models include simulation analysis and probabilistic risk assessment models.

functional analysis↗
NASA NTRSDate not supplied

Alternate Methods of Model Reduction to Avoid Dynamic Modal Truncation Error

Loads analysis is traditionally performed using dynamically reduced models, which provides the benefit to reduce run time. If the reduced model frequency cutoff is not chosen appropriately, the model will lack the dynamic content required to fully represent the response of the non-reduced model. Standard guidance for reduced model frequency content, provided in NASA-STD-5002, is to solve fixed base modes up to a minimum of 1.5x the model frequency content of interest and to employ static modal truncation methods such as residual vectors, the mode acceleration method, and the residual flexibility method to account for the truncated flexibility of the missing modes. Fixed base modes require the selection of a set of degrees of freedom to be constrained which, if not properly selected, may affect the accuracy of the reduced model by excluding some of the dynamic characteristic of the full model. In this case, the standard NASA guidance would be insufficient, but it may not be readily apparent that a portion of the reduced model response is missing. This error was encountered during an independent verification and validation (IV&V) effort, where it was observed that the resulting dynamic response was lower than the inline analysis. In this specific case, despite following the standard NASA model reduction guidelines in the selection of the frequency cutoff, the inline model still did not fully capture the necessary dynamic content. As part of the IV&V, an alternate reduction methodology was employed using an unconstrained mode acceleration method. The original model initially performed a constrained reduction to twice the frequency content of interest before doing a free-free run, employing the mode acceleration method to account for the truncated modes. In contrast in the IV&V, the unconstrained model reduced directly to the needed frequency content of the free-free run, avoiding any interactions between constraints and dynamic content. To verify the model, the original reduction methodology was used to generate a series of Hurty-Craig-Bampton reductions, each with a higher frequency cutoff than the previous. The results were shown to converge once the frequency cutoff increased past eight times the frequency content of interest. At the same frequency cutoff, the results of the Hurty-Craig-Bampton model converged with the results of the unconstrained mode acceleration model. This comparative study provided confidence that the results of the unconstrained modal acceleration reduced model were correct and that the Hurty-Craig-Bampton needed to increase its frequency cutoff to fully capture the dynamic response.

Erin Simmons↗
NASA NTRSDate not supplied

Alternate Methods of Model Reduction to Avoid Dynamic Modal Truncation Error

Loads analysis is traditionally performed using dynamically reduced models, which provides the benefit to reduce run time. If the reduced model frequency cutoff is not chosen appropriately, the model will lack the dynamic content required to fully represent the response of the non-reduced model. Standard guidance for reduced model frequency content, provided in NASA-STD-5002, is to solve fixed base modes up to a minimum of 1.5x the model frequency content of interest and to employ static modal truncation methods such as residual vectors, the mode acceleration method, and the residual flexibility method to account for the truncated flexibility of the missing modes. Fixed base modes require the selection of a set of degrees of freedom to be constrained which, if not properly selected, may affect the accuracy of the reduced model by excluding some of the dynamic characteristic of the full model. In this case, the standard NASA guidance would be insufficient, but it may not be readily apparent that a portion of the reduced model response is missing. This error was encountered during an independent verification and validation (IV&V) effort, where it was observed that the resulting dynamic response was lower than the inline analysis. In this specific case, despite following the standard NASA model reduction guidelines in the selection of the frequency cutoff, the inline model still did not fully capture the necessary dynamic content. As part of the IV&V, an alternate reduction methodology was employed using an unconstrained mode acceleration method. The original model initially performed a constrained reduction to twice the frequency content of interest before doing a free-free run, employing the mode acceleration method to account for the truncated modes. In contrast in the IV&V, the unconstrained model reduced directly to the needed frequency content of the free-free run, avoiding any interactions between constraints and dynamic content. To verify the model, the original reduction methodology was used to generate a series of Hurty-Craig-Bampton reductions, each with a higher frequency cutoff than the previous. The results were shown to converge once the frequency cutoff increased past eight times the frequency content of interest. At the same frequency cutoff, the results of the Hurty-Craig-Bampton model converged with the results of the unconstrained mode acceleration model. This comparative study provided confidence that the results of the unconstrained modal acceleration reduced model were correct and that the Hurty-Craig-Bampton needed to increase its frequency cutoff to fully capture the dynamic response.

Erin Simmons↗
NASA NTRS1984

Undetected error probability and throughput analysis of a concatenated coding scheme

The performance of a proposed concatenated coding scheme for error control on a NASA telecommand system is analyzed. In this scheme, the inner code is a distance-4 Hamming code used for both error correction and error detection. The outer code is a shortened distance-4 Hamming code used only for error detection. Interleaving is assumed between the inner and outer codes. A retransmission is requested if either the inner or outer code detects the presence of errors. Both the undetected error probability and the throughput of the system are analyzed. Results indicate that high throughputs and extremely low undetected error probabilities are achievable using this scheme.

Costello, D. J.↗