SEARCH · Engineering Papers

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Motion tomography via occupation kernels

The goal of motion tomography is to recover a description of a vector flow field using measurements along the trajectory of a sensing unit. In this paper, we develop a predictor corrector algorithm designed to recover vector flow fields from trajectory data with the use of occupation kernels developed by Rosenfeld et al. [9,10]. Specifically, we use the occupation kernels as an adaptive basis; that is, the trajectories defining our occupation kernels are iteratively updated to improve the estimation in the next stage. Initial estimates are established, then under mild assumptions, such as relatively straight trajectories, convergence is proven using the Contraction Mapping Theorem. We then compare the developed method with the established method by Chang et al. [5] by defining a set of error metrics. Here, we found that for simulated data, where a ground truth is available, our method offers a marked improvement over [5]. For a real-world example, where ground truth is not available, our results are similar results to the established method.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Using a surrogate-assisted Bayesian framework to calibrate the runoff-generation scheme in the Energy Exascale Earth System Model (E3SM) v1

Abstract. Runoff is a critical component of the terrestrial water cycle, and Earth system models (ESMs) are essential tools to study its spatiotemporal variability. Runoff schemes in ESMs typically include many parameters so that model calibration is necessary to improve the accuracy of simulated runoff. However, runoff calibration at a global scale is challenging because of the high computational cost and the lack of reliable observational datasets. In this study, we calibrated 11 runoff relevant parameters in the Energy Exascale Earth System Model (E3SM) Land Model (ELM) using a surrogate-assisted Bayesian framework. First, the polynomial chaos expansion machinery with Bayesian compressed sensing is used to construct computationally inexpensive surrogate models for ELM-simulated runoff at 0.5∘ × 0.5∘ for 1991–2010. The error metric between the ELM simulations and the benchmark data is selected to construct the surrogates, which facilitates efficient calibration and avoids the more conventional, but challenging, construction of high-dimensional surrogates for the ELM simulated runoff. Second, the Sobol' index sensitivity analysis is performed using the surrogate models to identify the most sensitive parameters, and our results show that, in most regions, ELM-simulated runoff is strongly sensitive to 3 of the 11 uncertain parameters. Third, a Bayesian method is used to infer the optimal values of the most sensitive parameters using an observation-based global runoff dataset as the benchmark. Our results show that model performance is significantly improved with the inferred parameter values. Although the parametric uncertainty of simulated runoff is reduced after the parameter inference, it remains comparable to the multimodel ensemble uncertainty represented by the global hydrological models in ISMIP2a. Additionally, the annual global runoff trend during the simulation period is not well constrained by the inferred parameter values, suggesting the importance of including parametric uncertainty in future runoff projections.

58 GEOSCIENCES↗

NASA NTRS1975

Automatic documentation system extension to multi-manufacturers' computers and to measure, improve, and predict software reliability

The DOMONIC system has been modified to run on the Univac 1108 and the CDC 6600 as well as the IBM 370 computer system. The DOMONIC monitor system has been implemented to gather data which can be used to optimize the DOMONIC system and to predict the reliability of software developed using DOMONIC. The areas of quality metrics, error characterization, program complexity, program testing, validation and verification are analyzed. A software reliability model for estimating program completion levels and one on which to base system acceptance have been developed. The DAVE system which performs flow analysis and error detection has been converted from the University of Colorado CDC 6400/6600 computer to the IBM 360/370 computer system for use with the DOMONIC system.

Simmons, D. B.↗

NASA NTRS1992

Robust Method For Robotic Mapping

Robot constructs map from experience. Topological model consists of nodes and arcs corresponding to distinctive places and local travel edges linking nearby distinctive places. Model created by linking places and edges. Enables accumulation of metrical information with reduced vulnerability to metrical errors. Applications include robotic sentires, robotic delivery trucks, robotic floor cleaners, and robotic lawnmowers.

Kuipers, Benjamin J.↗

NASA NTRS1993

Phase-retrieval algorithms for a complicated optical system

Phase-retrieval algorithms have been developed that handle a complicated optical system that requires multiple Fresnellike transforms to propagate from one end of the system to the other including the absorption by apertures in more than one plane and allowance for bad detector pixels. Gradient-search algorithms and generalizations of the iterative-transform phase-retrieval algorithms are derived. Analytic expressions for the gradient of an error metric, with respect to polynomial coefficients and with respect to point-by-point phase descriptions, are given. The entire gradient can be computed with the number of transforms required to propagate a wave front from one end of the optical system to the other and back again, independent of the number of coefficients or phase points. This greatly speeds the computation. The reconstruction of pupil amplitude is also given. A convergence proof of the generalized iterative transform algorithm is given. These improved algorithms permit a more accurate characterization of complicated optical systems from their point spread functions.

Fienup, J. R.↗

NASA NTRS1993

Nonlinear aerodynamic modeling using multivariate orthogonal functions

A technique was developed for global modeling of nonlinear aerodynamic coefficients using multivariate orthogonal functions based on the data. Each orthogonal function retained in the model was decomposed into an expansion of ordinary polynomials in the independent variables, so that the final model could be interpreted as selectively retained terms from a multivariable power series expansion. A predicted squared-error metric was used to determine the orthogonal functions to be retained in the model; analytical derivatives were easily computed. The approach was demonstrated on the Z-body axis aerodynamic force coefficient (Cz) wind tunnel data for an F-18 research vehicle which came from a tabular wind tunnel and covered the entire subsonic flight envelope. For a realistic case, the analytical model predicted experimental values of Cz very well. The modeling technique is shown to be capable of generating a compact, global analytical representation of nonlinear aerodynamics. The polynomial model has good predictive capability, global validity, and analytical differentiability.

Morelli, Eugene A.↗

NASA NTRS2009

Comparison of Image Restoration Methods for Lunar Epithermal Neutron Emission Mapping

Orbital measurements of neutrons by the Lunar Exploring Neutron Detector (LEND) onboard the Lunar Reconnaissance Orbiter are being used to quantify the spatial distribution of near surface hydrogen (H). Inferred H concentration maps have low signal-to-noise (SN) and image restoration (IR) techniques are being studied to enhance results. A single-blind. two-phase study is described in which four teams of researchers independently developed image restoration techniques optimized for LEND data. Synthetic lunar epithermal neutron emission maps were derived from LEND simulations. These data were used as ground truth to determine the relative quantitative performance of the IR methods vs. a default denoising (smoothing) technique. We review and used factors influencing orbital remote sensing of neutrons emitted from the lunar surface to develop a database of synthetic "true" maps for performance evaluation. A prior independent training phase was implemented for each technique to assure methods were optimized before the blind trial. Method performance was determined using several regional root-mean-square error metrics specific to epithermal signals of interest. Results indicate unbiased IR methods realize only small signal gains in most of the tested metrics. This suggests other physically based modeling assumptions are required to produce appreciable signal gains in similar low SN IR applications.

McClanahan, T. P.↗

NASA NTRS2012

An OSSE on Mesoscale Model Assimilation of Simulated HIRAD-Observed Hurricane Surface Winds

The hazards of landfalling hurricanes are well known, but progress on improving the intensity forecasts of these deadly storms at landfall has been slow. Many cite a lack of high-resolution data sets taken inside the core of a hurricane, and the lack of reliable measurements in extreme conditions near the surface of hurricanes, as possible reasons why even the most state-of-the-art forecasting models cannot seem to forecast intensity changes better. The Hurricane Imaging Radiometer (HIRAD) is a new airborne microwave remote sensor for observing hurricanes, and is operated and researched by NASA Marshall Space Flight Center in partnership with the NOAA Atlantic Oceanographic and Meteorological Laboratory/Hurricane Research Division, the University of Central Florida, the University of Michigan, and the University of Alabama in Huntsville. This instrument?s purpose is to study the wind field of a hurricane, specifically observing surface wind speeds and rain rates, in what has traditionally been the most difficult areas for other instruments to study; the high wind and heavy rain regions. Dr. T. N. Krishnamurti has studied various data assimilation techniques for hurricane and monsoon rain rates, and this study builds off of results obtained from utilizing his style of physical initializations of rainfall observations, but obtaining reliable observations in heavy rain regions has always presented trouble to our research of high-resolution rainfall forecasting. Reliable data from these regions at such a high resolution and wide swath as HIRAD provides is potentially very valuable to mesoscale forecasting of hurricane intensity. This study shows how the data assimilation technique of Ensemble Kalman Filtering (EnKF) in the Weather Research and Forecasting (WRF) model can be used to incorporate wind, and later rain rate, data into a mesoscale model forecast of hurricane intensity. The study makes use of an Observing System Simulation Experiment (OSSE) with a simulated HIRAD dataset sampled during a hurricane and uses EnKF to forecast the track and intensity prediction of the hurricane. Comparisons to truth and error metrics are used to assess the model?s forecast performance.

Albers, Cerese↗

NASA NTRS2017

Using Dispersed Modes During Model Correlation

The model correlation process for the modal characteristics of a launch vehicle is well established. After a test, parameters within the nominal model are adjusted to reflect structural dynamics revealed during testing. However, a full model correlation process for a complex structure can take months of man-hours and many computational resources. If the analyst only has weeks, or even days, of time in which to correlate the nominal model to the experimental results, then the traditional correlation process is not suitable. This paper describes using model dispersions to assist the model correlation process and decrease the overall cost of the process. The process creates thousands of model dispersions from the nominal model prior to the test and then compares each of them to the test data. Using mode shape and frequency error metrics, one dispersion is selected as the best match to the test data. This dispersion is further improved by using a commercial model correlation software. In the three examples shown in this paper, this dispersion based model correlation process performs well when compared to models correlated using traditional techniques and saves time in the post-test analysis.

Stewart, Eric C.↗

NASA NTRS2020

An AeroCom–AeroSat study: intercomparison of satellite AOD datasets for aerosol model evaluation

To better understand and characterize current uncertainties in the important observational constraint of climate models of aerosol optical depth (AOD), we evaluate and intercompare 14 satellite products, representing nine different retrieval algorithm families using observations from five different sensors on six different platforms. The satellite products (super-observations consisting of 1°×1° daily aggregated retrievals drawn from the years 2006, 2008 and 2010) are evaluated with AErosol RObotic NETwork (AERONET) and Maritime Aerosol Network (MAN) data. Results show that different products exhibit different regionally varying biases (both under- and overestimates) that may reach ±50 %, although a typical bias would be 15 %–25 % (depending on the product). In addition to these biases, the products exhibit random errors that can be 1.6 to 3 times as large. Most products show similar performance, although there are a few exceptions with either larger biases or larger random errors. The intercomparison of satellite products extends this analysis and provides spatial context to it. In particular, we show that aggregated satellite AOD agrees much better than the spatial coverage (often driven by cloud masks) within the 1°×1° grid cells. Up to ∼50 % of the difference between satellite AOD is attributed to cloud contamination. The diversity in AOD products shows clear spatial patterns and varies from 10 % (parts of the ocean) to 100 % (central Asia and Australia). More importantly, we show that the diversity may be used as an indication of AOD uncertainty, at least for the better performing products. This provides modellers with a global map of expected AOD uncertainty in satellite products, allows assessment of products away from AERONET sites, can provide guidance for future AERONET locations and offers suggestions for product improvements. We account for statistical and sampling noise in our analyses. Sampling noise, variations due to the evaluation of different subsets of the data, causes important changes in error metrics. The consequences of this noise term for product evaluation are discussed.

Aerosol↗

NASA NTRSDate not supplied

Autogenous Pressurization of a Cryogenic Tank Using Computational Fluid Dynamics

Well-validated simulation tools can be used to predict long-term storage and transfer of cryogenic propellants, which are essential to NASA’s mission plan to return to the Moon and continue to Mars. Autogenous pressurization of propellant tanks is used to supply rocket engine turbopumps with pressurized liquid fuel and oxidizer, preventing cavitation. Additionally, autogenous pressurization can be used to pressurize propellant tanks to support on-orbit propellant transfer. In lieu of expensive tests conducted on-orbit, accurate predictive computational models of these processes can be used to reduce system and propellant mass as well as mission risk. Using simulation tools also reduces the cost of analyzing and developing this technology. This study presents a multiphase computational fluid dynamics model capable of simulating autogenous pressurization of a large cryogenic tank using the commercial code STAR-CCM+. Experimental data from a full-scale tank pressurization test under terrestrial gravity is compared to simulation results using transient error metrics. Comparisons show good agreement and give confidence in using these and other validated simulation tools to develop cryogenic pressurization systems.

Cryogenics↗

NASA NTRSDate not supplied

Autogenous Pressurization of a Cryogenic Tank using Computational Fluid Dynamics

cryogenics↗

NASA NTRSDate not supplied

An L1 Adaptive Control Augmentation for a Lift Plus-Cruise Vehicle

This paper presents anL1adaptive control augmentation for a Lift-Plus-Cruise (L+C)vehicle. This class of vehicles operates in three flight modes with different dynamic behavior: vertical, transition, and forward flight. A robust uniform controller is used as a baseline to stabilize the system throughout these flight modes. The uniform controller is a linear control law designed around trim conditions of the aircraft and includes control allocation to achieve the desired forces and moments on the vehicle. TheL1control augmentation is designed for each of these trim conditions to compensate for the nonlinear time- and state-dependent uncertainties in the vehicle dynamics. The augmented control output is then added to the desired force and moment commands on the vehicle. Simulation results demonstrate the effectiveness of control augmentation for reducing the effects of unmodeled dynamics, reduced actuator effectiveness, and time-dependent disturbances. Effectiveness is demonstrated through tracking error metrics.

Andrew Patterson↗

NASA NTRS1970

MSFN unified S-band metric tracking capabilities

Error sources affecting metric tracking capability of unified S band

Salzberg, I.↗

DOE OSTI2021

Efficient flexible characterization of quantum processors with nested error models

We present a simple and powerful technique for finding a good error model for a quantum processor. The technique iteratively tests a nested sequence of models against data obtained from the processor, and keeps track of the best-fit model and its wildcard error (a metric of the amount of unmodeled error) at each step. Each best-fit model, along with a quantification of its unmodeled error, constitutes a characterization of the processor. We explain how quantum processor models can be compared with experimental data and to each other. We demonstrate the technique by using it to characterize a simulated noisy two-qubit processor.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRSDate not supplied

Fusing GeoNEX and VIIRS Surface BRDF Retrievals: Exploring a GEO-LEO Synergy

The Bidirectional Reflectance Distribution Function or BRDF, which describes the dependency of surface reflectance on the illumination-view geometries, are the foundation of many high-level satellite products for terrestrial and aquatic system monitoring. The latest geostationary sensors like GOES ABI provide high frequent (~10 minutes) observations of the Earth surface that feature continuously changing sun angles, allowing us to retrieve surface BRDF with dedicated atmospheric correction algorithms like MAIAC (Multi-Angle Implementation of Atmospheric Correction). For mid-latitude locations, because geostationary satellites have fixed view angles in the back-scattering directions, the angular sampling of surface BRDF by GEO sensors is not comprehensive. This study explores a GEO-LEO synergy to address this issue. We first extract concurrent GeoNEX and VIIRS BRDF data with the best quality (cloud-free and low aerosol loading) at chosen AERONET sites. We then compare the magnitude and the shape factors of the two set of BRDF parameters as well as their variations through the season. We calculate the “distances” between the GeoNEX and VIIRS BRDF by using them to cross-predict the top-of-atmosphere reflectance measured by their counterpart and evaluating the corresponding prediction errors. This metric allows us to derive a set of optimized BRDF parameters that minimize such distances or prediction errors, which are considered as the fused BRDF result. We validate the algorithm with reserved AERONET data and then apply it to generate the GEO-LEO BRDF synergy over CONUS. We expect the fused BRDF to have reduced uncertainties as compared to the source GeoNEX or VIIRS data and may find broadly application in deriving other high-level satellite products.

Geostationary satellite↗

DOE OSTI2022

Non-Parametric Statistical Analysis of Current Waveforms through Power System Sensors

The protection, control, and monitoring of the power grid is not possible without accurate measurement devices. As the percentage of renewable energy sources penetrating the existing grid infrastructure increases, so do uncertainties surrounding their effects on the everyday operation of the power system. Many of these devices are sources of high-frequency transients. These transients may be useful for identifying certain events or behaviors otherwise not seen in traditional analysis techniques. Therefore, the ability of sensors to accurately capture these phenomena is paramount. In this work, two commercial-grade power system distribution sensors are investigated in terms of their ability to replicate high-frequency phenomena by studying their responses to three events: a current inrush, a microgrid “close-in”, and a fault on the terminals of a wind turbine. Kernel density estimation is used to derive the non-parametric probability density functions of these error distributions and their adequateness is quantified utilizing the commonly used root mean square error (RMSE) metric. It is demonstrated that both sensors exhibit characteristics in the high harmonic range that go against the assumption that measurement error is normally distributed.

47 OTHER INSTRUMENTATION↗

NASA NTRS1991

Systematic methods for knowledge acquisition and expert system development

Nine cooperating rule-based systems, collectively called AUTOCREW, were designed to automate functions and decisions associated with a combat aircraft's subsystem. The organization of tasks within each system is described; performance metrics were developed to evaluate the workload of each rule base, and to assess the cooperation between the rule-bases. Each AUTOCREW subsystem is composed of several expert systems that perform specific tasks. AUTOCREW's NAVIGATOR was analyzed in detail to understand the difficulties involved in designing the system and to identify tools and methodologies that ease development. The NAVIGATOR determines optimal navigation strategies from a set of available sensors. A Navigation Sensor Management (NSM) expert system was systematically designed from Kalman filter covariance data; four ground-based, a satellite-based, and two on-board INS-aiding sensors were modeled and simulated to aid an INS. The NSM Expert was developed using the Analysis of Variance (ANOVA) and the ID3 algorithm. Navigation strategy selection is based on an RSS position error decision metric, which is computed from the covariance data. Results show that the NSM Expert predicts position error correctly between 45 and 100 percent of the time for a specified navaid configuration and aircraft trajectory. The NSM Expert adapts to new situations, and provides reasonable estimates of hybrid performance. The systematic nature of the ANOVA/ID3 method makes it broadly applicable to expert system design when experimental or simulation data is available.

Belkin, Brenda L.↗