SEARCH · Engineering Papers

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

All sources NASA NTRS DOE OSTI

NASA NTRS2017

Annoyance to Noise Produced by a Distributed Electric Propulsion High-Lift System

A psychoacoustic test was performed using simulated sounds from a distributed electric propulsion aircraft concept to help understand factors associated with human annoyance. A design space spanning the number of high-lift leading edge propellers and their relative operating speeds, inclusive of time varying effects associated with motor controller error and atmospheric turbulence, was considered. It was found that the mean annoyance response varies in a statistically significant manner with the number of propellers and with the inclusion of time varying effects, but does not differ significantly with the relative RPM between propellers. An annoyance model was developed, inclusive of confidence intervals, using the noise metrics of loudness, roughness, and tonality as predictors.

Rizzi, Stephen A.↗

NASA NTRS2003

A new method to evaluate human-robot system performance

One of the key issues in space exploration is that of deciding what space tasks are best done with humans, with robots, or a suitable combination of each. In general, human and robot skills are complementary. Humans provide as yet unmatched capabilities to perceive, think, and act when faced with anomalies and unforeseen events, but there can be huge potential risks to human safety in getting these benefits. Robots provide complementary skills in being able to work in extremely risky environments, but their ability to perceive, think, and act by themselves is currently not error-free, although these capabilities are continually improving with the emergence of new technologies. Substantial past experience validates these generally qualitative notions. However, there is a need for more rigorously systematic evaluation of human and robot roles, in order to optimize the design and performance of human-robot system architectures using well-defined performance evaluation metrics. This article summarizes a new analytical method to conduct such quantitative evaluations. While the article focuses on evaluating human-robot systems, the method is generally applicable to a much broader class of systems whose performance needs to be evaluated.

Space Flight/instrumentation↗

NASA NTRS2014

GCR Environmental Models III: GCR Model Validation and Propagated Uncertainties in Effective Dose

This is the last of three papers focused on quantifying the uncertainty associated with galactic cosmic rays (GCR) models used for space radiation shielding applications. In the first paper, it was found that GCR ions with Z>2 and boundary energy below 500 MeV/nucleon induce less than 5% of the total effective dose behind shielding. This is an important finding since GCR model development and validation have been heavily biased toward Advanced Composition Explorer/Cosmic Ray Isotope Spectrometer measurements below 500 MeV/nucleon. Weights were also developed that quantify the relative contribution of defined GCR energy and charge groups to effective dose behind shielding. In the second paper, it was shown that these weights could be used to efficiently propagate GCR model uncertainties into effective dose behind shielding. In this work, uncertainties are quantified for a few commonly used GCR models. A validation metric is developed that accounts for measurements uncertainty, and the metric is coupled to the fast uncertainty propagation method. For this work, the Badhwar-O'Neill (BON) 2010 and 2011 and the Matthia GCR models are compared to an extensive measurement database. It is shown that BON2011 systematically overestimates heavy ion fluxes in the range 0.5-4 GeV/nucleon. The BON2010 and BON2011 also show moderate and large errors in reproducing past solar activity near the 2000 solar maximum and 2010 solar minimum. It is found that all three models induce relative errors in effective dose in the interval [-20%, 20%] at a 68% confidence level. The BON2010 and Matthia models are found to have similar overall uncertainty estimates and are preferred for space radiation shielding applications.

Slaba, Tony C.↗

NASA NTRS2012

A Change Impact Analysis to Characterize Evolving Program Behaviors

Change impact analysis techniques estimate the potential effects of changes made to software. Directed Incremental Symbolic Execution (DiSE) is an intraprocedural technique for characterizing the impact of software changes on program behaviors. DiSE first estimates the impact of the changes on the source code using program slicing techniques, and then uses the impact sets to guide symbolic execution to generate path conditions that characterize impacted program behaviors. DiSE, however, cannot reason about the flow of impact between methods and will fail to generate path conditions for certain impacted program behaviors. In this work, we present iDiSE, an extension to DiSE that performs an interprocedural analysis. iDiSE combines static and dynamic calling context information to efficiently generate impacted program behaviors across calling contexts. Information about impacted program behaviors is useful for testing, verification, and debugging of evolving programs. We present a case-study of our implementation of the iDiSE algorithm to demonstrate its efficiency at computing impacted program behaviors. Traditional notions of coverage are insufficient for characterizing the testing efforts used to validate evolving program behaviors because they do not take into account the impact of changes to the code. In this work we present novel definitions of impacted coverage metrics that are useful for evaluating the testing effort required to test evolving programs. We then describe how the notions of impacted coverage can be used to configure techniques such as DiSE and iDiSE in order to support regression testing related tasks. We also discuss how DiSE and iDiSE can be configured for debugging finding the root cause of errors introduced by changes made to the code. In our empirical evaluation we demonstrate that the configurations of DiSE and iDiSE can be used to support various software maintenance tasks

Rungta, Neha Shyam↗

NASA NTRS2018

Estimation of Lower-Body Kinetics from Loading Profile and Kinematics Alone, Without Measured Ground Reaction Forces

Biomechanical models of human motion can estimate kinetic outcomes, such as joint moments, joint forces and muscle forces. Typically, one performs an inverse dynamics (ID) analysis to compute joint moments from joint angles and measured external forces. Sometimes it is impractical to measure ground reaction forces and moments (GRF&M). We devised an empirical method for performing ID analysis of resistance exercises without measured GRF&M. The method solves the multibody dynamics equations of motion with four key assumptions about the GRF&M that reduce the number of unknowns. The assumptions are 1) negligible ground reaction moments, 2) fixed lateral/medial location of the center of pressure (COP), 3) equal fore/aft location of the COP between the feet, and 4) constant angle of the GRF vector relative to the vertical axis in the frontal plane. We used evaluation trials from a spaceflight countermeasure resistance training device to test this approach. Four participants performed squat and deadlift exercises at various loads. We compared results from traditional ID analysis to results without measured GRF&M using our method. We found that joint moment trajectories in the sagittal plane were qualitatively similar in shape between the two methods, and the amount of root mean squared error (RMSE), measured by difference in joint moment impulse, was typically under 10 percent. Non-sagittal joint moment trajectories, which are much lower in overall magnitude, were not qualitatively similar in shape between the two methods. Non-sagittal moments displayed much higher RMSE, with typical values well over 50 percent. These findings were further supported by validation metrics (Sprague and Geers' P and M metrics, Pearson's r correlation coefficient). Based on these findings, we concluded that useful kinetic results are obtained from ID analysis of squat and deadlift exercises, even when GRF&M are not measured, as long as the outcomes of interest lie in the sagittal plane.

computational modeling↗

NASA NTRSDate not supplied

Cartesian Mesh Simulations for the Third AIAA Sonic Boom Prediction Workshop

Simulation results are presented for all cases from the Third AIAA Sonic Boom Prediction Workshop. An inviscid, embedded-boundary Cartesian-mesh flow solver is used in conjunction with adjoint-based mesh adaptation to compute nearfield pressure signatures. Specialized techniques are applied to maximize accuracy and minimize cost on Cartesian meshes. The Richardson-based error estimate highlights regions of the signatures most sensitive to mesh refinement. Timing results and coarse, medium, and fine mesh sizes for nearfield cases demonstrate that the parallel decomposition approach is efficient in both computational time and wall-clock. Pressure signals are propagated to the ground using an augmented Burgers’ equation solver to predict boom carpets. Ground signatures and loudness metrics are presented for a standard atmosphere as well as more realistic atmospheric profiles, which affect overall noise levels and can significantly widen the boom carpet. Mesh convergence studies show that high sampling frequencies, around 500 kHz, are required for propagation, and the sampling frequency increases at large off-track angles with longer acoustic ray paths and propagation times. The numerical methods yield accurate results for predicting low sonic boom signatures while being among the least computationally expensive of the workshop.

ARMD↗

NASA NTRS1997

Foundations for Measuring Volume Rendering Quality

The goal of this paper is to provide a foundation for objectively comparing volume rendered images. The key elements of the foundation are: (1) a rigorous specification of all the parameters that need to be specified to define the conditions under which a volume rendered image is generated; (2) a methodology for difference classification, including a suite of functions or metrics to quantify and classify the difference between two volume rendered images that will support an analysis of the relative importance of particular differences. The results of this method can be used to study the changes caused by modifying particular parameter values, to compare and quantify changes between images of similar data sets rendered in the same way, and even to detect errors in the design, implementation or modification of a volume rendering system. If one has a benchmark image, for example one created by a high accuracy volume rendering system, the method can be used to evaluate the accuracy of a given image.

Williams, Peter L.↗

NASA NTRS2011

Orthographic Stereo Correlator on the Terrain Model for Apollo Metric Images

A stereo correlation method on the object domain is proposed to generate the accurate and dense Digital Elevation Models (DEMs) from lunar orbital imagery. The NASA Ames Intelligent Robotics Group (IRG) aims to produce high-quality terrain reconstructions of the Moon from Apollo Metric Camera (AMC) data. In particular, IRG makes use of a stereo vision process, the Ames Stereo Pipeline (ASP), to automatically generate DEMs from consecutive AMC image pairs. Given camera parameters of an image pair from bundle adjustment in ASP, a correlation window is defined on the terrain with the predefined surface normal of a post rather than image domain. The squared error of back-projected images on the local terrain is minimized with respect to the post elevation. This single dimensional optimization is solved efficiently and improves the accuracy of the elevation estimate.

Terrain Model↗

NASA NTRS2019

On the Use of Optimization Techniques for Turbulence Model Calibration

This article discusses the use of numerical optimization procedures to aid in the calibration of turbulence model coefficients. Such methods would increase the rigor and repeatability of the calibration procedure by requiring clearly defined and objective optimization metrics, and could be used to identify unique combinations of coefficient values for specific flow problems. The approach is applied to the re-calibration of an explicit algebraic Reynolds stress model for the incompressible planar mixing layer using the Nelder-Mead simplex algorithm and a micro-genetic algorithm with minimally imposed constraints. Three composite fitness functions, each based upon the error in the mixing layer growth rate and the normal and shear components of the Reynolds stresses, are investigated. The results demonstrate a significant improvement in the target objectives through the adjustment of three pressure-strain coefficients. Adjustments of additional coefficients provide little further benefit. Issues regarding the effectiveness of the fitness functions and the efficiency of the optimization algorithms are also discussed.

Yoder, Dennis A.↗

NASA NTRSDate not supplied

Towards Understanding Data Requirements for Developing Automatic Speech Recognition Systems for Air Traffic Control

In recent years, the application of automatic speech recognition has gained popularity across diverse industries, including aviation. Given the many applications focusing on transcribing air traffic control and management communication, this paper explores the training of OpenAI's Whisper model across multiple existing public and private air traffic control voice datasets in an effort to improve robustness. Combining roughly 60+ hours of various air traffic datasets, our goal is to train a unified Whisper model and expect an average word error rate reduction across testing datasets. Furthermore, this work aims to understand the data quantity requirements for achieving state-of-the art results by comprehensively training Whisper on varying dataset sizes. This work has the potential to improve automatic speech recognition performance across the domain, improve understanding of the quantity of data required by an aviation speech recognition system, and lastly provide metrics to compare and improve upon in future research.

ATC↗

NASA NTRSDate not supplied

Use of Beacon Satellites for Efficient Uplink Transmission for Free-Space Quantum Entanglement Distribution

Efficient transmission of optical beams from ground to space is important for free-space entanglement distribution in a dual-uplink architecture in which photons from entanglement sources at two ground locations interact in an optical Bell-state measurement implemented on a satellite. Efficient transmission requires large transmitting apertures to minimize diffraction losses and high-performance adaptive optics to overcome the effects of atmospheric turbulence. Achieving good atmospheric compensation for beams transmitted to satellites is challenging because the need to point ahead of the satellite introduces anisoplanatism error that limits the effectiveness of the adaptive optics. An attractive approach for solving the point-ahead problem is to position a separate small satellite at the point-ahead location to provide a beacon source for the adaptive optics. This paper presents analysis of the requirements for the beacon source(s) for a dual-uplink entanglement-distribution demonstration. A relative performance metric is defined and evaluated for multiple beacon configurations, from a single, fixed beacon serving both ground sites to a dedicated beacon for each site continually adjusted to be at the correct point-ahead distance from the target satellite.

quantum↗

NASA NTRS2009

Comparison of Node-Centered and Cell-Centered Unstructured Finite-Volume Discretizations: Viscous Fluxes - Part 1

Discretization of the viscous terms in current finite-volume unstructured-grid schemes are compared using node-centered and cell-centered approaches in two dimensions. Accuracy and efficiency are studied for six nominally second-order accurate schemes: a node-centered scheme, cell-centered node-averaging schemes with and without clipping, and cell-centered schemes with unweighted, weighted, and approximately mapped least-square face gradient reconstruction. The grids considered range from structured (regular) grids to irregular grids composed of arbitrary mixtures of triangles and quadrilaterals, including random perturbations of the grid points to bring out the worst possible behavior of the solution. Two classes of tests are considered. The first class of tests involves smooth manufactured solutions on both isotropic and highly anisotropic grids with discontinuous metrics, typical of those encountered in grid adaptation. The second class concerns solutions and grids varying strongly anisotropically over a curved body, typical of those encountered in high-Reynolds number turbulent flow simulations. Results from the first class indicate the face least-square methods, the node-averaging method without clipping, and the node-centered method demonstrate second-order convergence of discretization errors with very similar accuracies per degree of freedom. The second class of tests are more discriminating. The node-centered scheme is always second order with an accuracy and complexity in linearization comparable to the best of the cell-centered schemes. In comparison, the cell-centered node-averaging schemes are less accurate, have a higher complexity in linearization, and can fail to converge to the exact solution when clipping of the node-averaged values is used. The cell-centered schemes using least-square face gradient reconstruction have more compact stencils with a complexity similar to the complexity of the node-centered scheme. For simulations on highly anisotropic curved grids, the least-square methods have to be amended either by introducing a local mapping of the surface anisotropy or modifying the scheme stencil to reflect the direction of strong coupling.

Diskin, Boris↗

NASA NTRS2010

Comparison of Node-Centered and Cell-Centered Unstructured Finite-Volume Discretizations: Viscous Fluxes

Discretization of the viscous terms in current finite-volume unstructured-grid schemes are compared using node-centered and cell-centered approaches in two dimensions. Accuracy and complexity are studied for four nominally second-order accurate schemes: a node-centered scheme and three cell-centered schemes - a node-averaging scheme and two schemes with nearest-neighbor and adaptive compact stencils for least-square face gradient reconstruction. The grids considered range from structured (regular) grids to irregular grids composed of arbitrary mixtures of triangles and quadrilaterals, including random perturbations of the grid points to bring out the worst possible behavior of the solution. Two classes of tests are considered. The first class of tests involves smooth manufactured solutions on both isotropic and highly anisotropic grids with discontinuous metrics, typical of those encountered in grid adaptation. The second class concerns solutions and grids varying strongly anisotropically over a curved body, typical of those encountered in high-Reynolds number turbulent flow simulations. Tests from the first class indicate the face least-square methods, the node-averaging method without clipping, and the node-centered method demonstrate second-order convergence of discretization errors with very similar accuracies per degree of freedom. The tests of the second class are more discriminating. The node-centered scheme is always second order with an accuracy and complexity in linearization comparable to the best of the cell-centered schemes. In comparison, the cell-centered node-averaging schemes may degenerate on mixed grids, have a higher complexity in linearization, and can fail to converge to the exact solution when clipping of the node-averaged values is used. The cell-centered schemes using least-square face gradient reconstruction have more compact stencils with a complexity similar to that of the node-centered scheme. For simulations on highly anisotropic curved grids, the least-square methods have to be amended either by introducing a local mapping based on a distance function commonly available in practical schemes or modifying the scheme stencil to reflect the direction of strong coupling. The major conclusion is that accuracies of the node centered and the best cell-centered schemes are comparable at equivalent number of degrees of freedom.

Diskin, Boris↗

NASA NTRS1986

Precise interferometric tracking of the DSCS II geosynchronous orbiter

A demonstration of the precise tracking of a geosynchronous orbiter by radio metric techniques based on very-long-baseline interferometry (VLBI) has been jointly conducted by the Jet Propulsion Laboratory and Japan's Radio Research Laboratory. Simultaneous observations of a U.S. Air Force communications satellite from tracking stations in California, Australia, and Japan have determined the satellite's position with an accuracy of a few meters. Accuracy claims are based on formal statistics, which include the effects of errors in non-estimated parameters and which are supported by a chi-squared of less than one, and on the consistency of orbit solutions from disjoint data sets. A study made to assess the impact of shorter baselines and reduced data noise concludes that with a properly designed system, similar accuracy could be obtained for either a satellite viewed from stations located within the continental U.S. or for a satellite viewed from stations within Japanese territory.

Border, J. S.↗

NASA NTRS2011

Automatic Tie Pointer for In-Situ Pointing Correction

The MARSAUTOTIE program generates tie points for use with the Mars pointing correction software "In-Situ Pointing Correction and Rover Microlocalization," (NPO-46696) Soft ware Tech Briefs, Vol. 34, No. 9 (September 2010), page 18, in a completely automated manner, with no operator intervention. It takes the place of MARSTIE, although MARSTIE can be used to interactively edit the tie points afterwards. These tie points are used to create a mosaic whose seams (boundaries of input images) have been geometrically corrected to reduce or eliminate errors and mis-registrations. The methods used to find appropriate tie points for use in creating a mosaic are unique, having been designed to work in concert with the "MARSNAV" program to be most effective in reducing or eliminating geometric seams in a mosaic. The program takes the input images and finds overlaps according to the nominal pointing. It then finds the most interesting areas using a scene activity metric. Points with higher scene activity are more likely to correlate successfully in the next step. It then uses correlation techniques to find matching points in the overlapped image. Finally, it performs a series of steps to reduce the number of tie points to a manageable level. These steps incorporate a number of heuristics that have been devised using experience gathered by tie pointing mosaics manually during MER operations. The software makes use of the PIG library as described in "Planetary Image Geometry Library" (NPO-46658), NASA Tech Briefs, Vol. 34, No. 12 (December 2010), page 30, so it is multi-mission, applicable without change to any in-situ mission supported by PIG. The MARSAUTOTIE algorithm is automated, so it requires no user intervention. Although at the time of this reporting it has not been done, this program should be suitable for integration into a fully automated mosaic production pipeline.

Deen, Robert G/↗

NASA NTRS2016

Quantitative Validation of the Integrated Medical Model (IMM) for ISS Missions

Lifetime Surveillance of Astronaut Health (LSAH) provided observed medical event data on 33 ISS and 111 STS person-missions for use in further improving and validating the Integrated Medical Model (IMM). Using only the crew characteristics from these observed missions, the newest development version, IMM v4.0, will simulate these missions to predict medical events and outcomes. Comparing IMM predictions to the actual observed medical event counts will provide external validation and identify areas of possible improvement. In an effort to improve the power of detecting differences in this validation study, the total over each program ISS and STS will serve as the main quantitative comparison objective, specifically the following parameters: total medical events (TME), probability of loss of crew life (LOCL), and probability of evacuation (EVAC). Scatter plots of observed versus median predicted TMEs (with error bars reflecting the simulation intervals) will graphically display comparisons while linear regression will serve as the statistical test of agreement. Two scatter plots will be analyzed 1) where each point reflects a mission and 2) where each point reflects a condition-specific total number of occurrences. The coefficient of determination (R2) resulting from a linear regression with no intercept bias (intercept fixed at zero) will serve as an overall metric of agreement between IMM and the real world system (RWS). In an effort to identify as many possible discrepancies as possible for further inspection, the -level for all statistical tests comparing IMM predictions to observed data will be set to 0.1. This less stringent criterion, along with the multiple testing being conducted, should detect all perceived differences including many false positive signals resulting from random variation. The results of these analyses will reveal areas of the model requiring adjustment to improve overall IMM output, which will thereby provide better decision support for mission critical applications.

validation↗

NASA NTRSDate not supplied

Growth Curve Parameterization of Metabolic Activity of Yeast Cells for BioSentinel

The goal of the BioSentinel small satellite payload is to measure the effect of deep space radiation on the growth and metabolic activity of yeast cells. Raw test data is generated by fluidics cards containing yeast cells rehydrated at different periods, with metabolic activity measured by the reduction of alamarBlue. Each card well has a sensor array that measures the amount of red, green, and infrared light transmitted through the yeast culture. This illumination data is then converted to absorbance values, which are further converted into concentrations. The ultimate objective is to convert these concentrations into biologically-relevant metrics that can be compared against one another to determine changes due to differential radiation exposure. Beginning with IR absorbance data (corresponding to cell density) from ground studies, three parameters from a sigmoidal growth curve were extracted and analyzed: 𝜆 (lag phase), 𝜇 (max growth rate), and A (max cell growth). The data was fit to the Gompertz model of microbial growth using non-linear regression (Minitab), as the fit error was reduced compared to the simpler logistic growth curve. Graphs showed that the data contained a discrepancy (drift) in the lag phase that is attributable to a slow, constant loss of moisture. Correcting this discrepancy by fitting the first 25 hours of the data to a power function and subtracting these values from the absorbance readings obtained a better statistical fit to the growth curve in the lag phase. A power fit was selected over a linear fit because it reflected the effects of constant volume loss. This correction to the BioSentinel data analysis pipeline will enable quantitative statistical analysis of the effect of different levels of deep space radiation on yeast cells. Future work includes automation of drift correction and curve modeling to extract these parameters directly from data.

Growth Curve↗

NASA NTRSDate not supplied

Comparison Study of Machine Learning Techniques to Predict Flight Energy Consumption for Advanced Air Mobility

This paper addresses the need to predict the flight energy consumption of aerial vehicles in the presence of wind using machine learning techniques. The presented work is critical to achieving sustainable and efficient operations for Advanced Air Mobility (AAM) and to evaluating the readiness of the ground-supporting energy infrastructure, e.g., electric grid and AAM portals. The flight energy consumption is described using the "energy per meter" (EPM) metric. We present a comparison study of influential machine learning techniques in predicting EPM using real-world flight test data. We presented new results of using the Decision Tree, Random Forest, and linear regression techniques, along with our previous results using the Recurrent Neural Network and Feed Forward Neural Network techniques. The comparison results show that the Linear Regression method outperforms other methods on the basis of the Mean Squared Error and error variance.

Machine Learning↗