Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “error metric”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Assessment of the ParFlow–CLM CONUS 1.0 integrated hydrologic model: evaluation of hyper-resolution water balance components across the contiguous United States

Abstract. Recent advancements in computational efficiency and Earth system modeling have awarded hydrologists with increasingly high-resolution models of terrestrial hydrology, which are paramount to understanding and predicting complex fluxes of moisture and energy. Continental-scale hydrologic simulations are, in particular, of interest to the hydrologic community for numerous societal, scientific, and operational benefits. The coupled hydrology–land surface model ParFlow–CLM configured over the continental United States (PFCONUS) has been employed in previous literature to study scale-dependent connections between water table depth, topography, recharge, and evapotranspiration, as well as to explore impacts of anthropogenic aquifer depletion to the water and energy balance. These studies have allowed for an unprecedented process-based understanding of the continental water cycle at high resolution. Here, we provide the most comprehensive evaluation of PFCONUS version 1.0 (PFCONUSv1) performance to date by comparing numerous modeled water balance components with thousands of in situ observations and several remote sensing products and using a range of statistical performance metrics for evaluation. PFCONUSv1 comparisons with these datasets are a promising indicator of model fidelity and ability to reproduce the continental-scale water balance at high resolution. Areas for improvement are identified, such as a positive streamflow bias at gauges in the eastern Great Plains, a shallow water table bias over many areas of the model domain, and low bias in seasonal total water storage amplitude, especially for the Ohio, Missouri, and Arkansas River basins. We discuss several potential sources for model bias and suggest that minimizing error in topographic processing and meteorological forcing would considerably improve model performance. Results here provide a benchmark and guidance for further PFCONUS model development, and they highlight the importance of concurrently evaluating all hydrologic components and fluxes to provide a multivariate, holistic validation of the complete modeled water balance.

O'Neill, Mary M. F. (ORCID:0000000221455165)↗
NASA NTRS2016

Wind Information Uplink to Aircraft Performing Interval Management Operations

Interval Management (IM) is an ADS-B-enabled suite of applications that use ground and flight deck capabilities and procedures designed to support the relative spacing of aircraft (Barmore et al., 2004, Murdoch et al. 2009, Barmore 2009, Swieringa et al. 2011; Weitz et al. 2012). Relative spacing refers to managing the position of one aircraft to a time or distance relative to another aircraft, as opposed to a static reference point such as a point over the ground or clock time. This results in improved inter-aircraft spacing precision and is expected to allow aircraft to be spaced closer to the applicable separation standard than current operations. Consequently, if the reduced spacing is used in scheduling, IM can reduce the time interval between the first and last aircraft in an overall arrival flow, resulting in increased throughput. Because IM relies on speed changes to achieve precise spacing, it can reduce costly, low-altitude, vectoring, which increases both efficiency and throughput in capacity-constrained airspace without negatively impacting controller workload and task complexity. This is expected to increase overall system efficiency. The Flight Deck Interval Management (FIM) equipment provides speeds to the flight crew that will deliver them to the achieve-by point at the controller-specified time, i.e., assigned spacing goal, after the target aircraft crosses the achieve-by point (Figure 1.1). Since the IM and target aircraft may not be on the same arrival procedure, the FIM equipment predicts the estimated times of arrival (ETA) for both the IM and target aircraft to the achieve-by point. This involves generating an approximate four-dimensional trajectory for each aircraft. The accuracy of the wind data used to generate those trajectories is critical to the success of the IM operation. There are two main forms of uncertainty in the wind information used by the FIM equipment. The first is the accuracy of the forecast modeling done by the weather provider. This is generally a global environmental prediction obtained from a weather model such as the Rapid Refresh (RAP) from the National Centers for Environmental Prediction (NCEP). The weather forecast data will have errors relative to the actual, or truth, winds that the aircraft will encounter. The second source of uncertainty is that only a small subset of the forecast data can be uplinked to the aircraft for use by the FIM equipment. This results in loss of additional information. The Federal Aviation Administration (FAA) and RTCA are currently developing standards for the communication of wind and atmospheric data to the aircraft for use in NextGen operations. This study examines the impact of various wind forecast sampling methods on IM performance metrics to inform the standards development.

Ahmad, Nashat N.↗
NASA NTRS2017

Program to Optimize Simulated Trajectories II (POST2) Surrogate Models for Mars Ascent Vehicle (MAV) Performance Assessment

The primary purpose of the multiPOST tool is to enable the execution of much larger sets of vehicle cases to allow for broader trade space exploration. However, this exploration is not achieved solely with the increased case throughput. The multiPOST tool is applied to carry out a Design of Experiments (DOE), which is a set of cases that have been structured to capture a maximum amount of information about the design space with minimal computational effort. The results of the DOE are then used to fit a surrogate model, ultimately enabling parametric design space exploration. The approach used for the MAV study includes both DOE and surrogate modeling. First, the primary design considerations for the vehicle were used to develop the variables and ranges for the multiPOST DOE. The final set of DOE variables were carefully selected in order to capture the desired vehicle trades and take into account any special considerations for surrogate modeling. Next, the DOE sets were executed through multiPOST. Following successful completion of the DOE cases, a manual verification trial was performed. The trial involved randomly selecting cases from the DOE set and running them by hand. The results from the human analyst's run and multiPOST were then compared to ensure that the automated runs were being executed properly. Completion of the verification trials was then followed by surrogate model fitting. After fits to the multiPOST data were successfully created, the surrogate models were used as a stand-in for POST2 to carry out the desired MAV trades. Using the surrogate models in lieu of POST2 allowed for visualization of vehicle sensitivities to the input variables as well as rapid evaluation of vehicle performance. Although the models introduce some error into the output of the trade study, they were very effective at identifying areas of interest within the trade space for further refinement by human analysts. The next section will cover all of the ground rules and assumptions associated with DOE setup and multiPOST execution. Section 3.1 gives the final DOE variables and ranges, while section 3.2 addresses the POST2 specific assumptions. The results of the verification trials are given in section 4. Section 5 gives the surrogate model fitting results, including the goodness-of-fit metrics for each fit. Finally, the MAV specific results are discussed in section 6.

Zwack, M. R.↗
NASA NTRS2021

Chapter 5: Extravehicular Activity Metabolic Rate Model: Metabolic Rate Estimated from Heart Rate

In-flight monitoring of crew metabolic rates during extravehicular activity (EVA) provides crucial information in mitigating injury. The purpose of this study was to investigate the relationship of crewmember heart rate (HR) and metabolic rate (MR) during EVA operations to develop a predictive linear model. HR and MR data was collected from 132 EVAs from Shuttle and International Space Station (ISS) missions. MR was collected every 2-min from portable life support system delta oxygen decay, while HR was collected every 20-sec via electrocardiogram. HR was down sampled to every 2-min to match MR during EVA for evaluation. Further, a new metric was observed from direct relations between metabolic rate with HR over EVA time (MR/HR) measured as a BTU/beat. A range of BTU/beat was collected as a conversion scale between MR and HR at different EVA workloads categorized by increased MR. Both HR and MR values were observed to decrease through the duration of EVA. Similarly, HR and MR slopes decreased at start of EVA compared to end of EVA. MR/HR values were used to predict MR from HR over the entire duration of EVA with root mean square error less than 200 BTU/Hr. Additionally, MR was predicted based on HR values during EVA via a calculated simple linear regression. A regression equation was found for each EVA drawing relations between HR and MR (F (2923.84) and P<0.0001) with an R(sup 2) value of 0.402. Individual crew regressions improved prediction and R(sup 2) to greater than 0.82. Two models are presented to determine metabolic rate from heart rate during EVA. Results draw correlations for heart rate and metabolic rate fluctuations during EVA for individualized crew predictions during future operations. The linear models correlate to Apollo prediction data during historic EVAs.

Metabolic rate predictions↗
DOE OSTI2023

Retrieval of temperature and humidity profiles from ground-based high-resolution infrared observations using an adaptive fast iterative algorithm

Various retrieval algorithms have been developed for retrieving temperature and water vapor profiles from Atmospheric Emitted Radiance Interferometer (AERI) observations. The physical retrieval algorithm, named AERI Optimal Estimation (AERIoe), outperforms other retrieval algorithms in many aspects except the retrieval time, which is significantly increased due to the complex radiative transfer process. The calculation of the Jacobian matrix is the most computationally intensive step of the physical retrieval algorithm. Interestingly, an analysis of the change in AERI observations' information content with respect to Jacobians revealed that the AERIoe algorithm's performance presents negligible dependence on these metrics. Thus, the Jacobian matrix could remain unchanged when the variation in the atmospheric state is small in the retrieval process to reduce the most time-consuming computation. On the basis of the above findings, a fast physical–iterative retrieval algorithm was proposed by adaptively recalculating Jacobians in keeping with the changes in the atmospheric state. Experiments with synthetic observations demonstrate that the proposed method experiences an average reduction in retrieval time by an impressive 59 % compared to the original AERIoe algorithm while achieving maximum root-mean-square errors of less than 0.95 K and 0.22 log(ppmv) for heights below 3 km for the temperature and water vapor profile, respectively. Further analyses revealed that the fast-retrieval algorithm reached an acceptable convergence rate of 98.7 %, marginally lower than AERIoe's 99.9 % convergence rate for the 826 cases used in this study.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1992

Principled halftoning based on human vision models

When models of human vision adequately measure the relative quality of candidate halftonings of an image, the problem of halftoning the image becomes equivalent to the search problem of finding a halftone that optimizes the quality metric. Because of the vast number of possible halftones, and the complexity of image quality measures, this principled approach has usually been put aside in favor of fast algorithms that seem to perform well. We find that the principled approach can lead to a range of useful halftoning algorithms, as we trade off speed for quality by varying the complexity of the quality measure and the thoroughness of the search. High quality halftones can be obtained reasonably quickly, for example, by using as a measure the vector length of the error image filtered by a contrast sensitivity function, and, as the search procedure, the sequential adjustment of individual pixels to improve the quality measure. If computational resources permit, simulated annealing can find nearly optimal solutions.

Mulligan, Jeffrey B.↗
DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as time series data shifts. In this research, a changepoint detection (CPD) algorithm that automatically detects data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window across the labeled data sets. Pending approval, we plan to release the labeled data sets for this research on NREL's DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package. By supplying the training sets and algorithm, we hope to encourage further development in this research space.

data shift↗
NASA NTRS2019

Health Management and Prognostics for Electric Aircraft Powertrain

W and c Any air borne vehicle needs incorporating safety as key parameter of measure, and inclusion of autonomy raises the critical need for safety under autonomous operations. Management of faults and component degradation is key as complexity in autonomous operations grow over the period of time. Therefore, in addition to basic operational requirements, an autonomous electric vehicle should be able to make accurate estimates of its current system health and take the correct decisions to complete its mission successfully. Real-time safety and state-awareness tools are therefore essential for the vehicle to be able to reach its destination in a safe and successful manner. The need for safety assurance and health management capabilities is particularly relevant for aircraft electric propulsion systems, which are relatively new and with limited historical to learn. They are critical systems requiring high power density along with reliability, resilience, efficient management of weight, and operational costs. A model- based fault diagnosis and prognostics approach of complex critical systems can successfully accomplish the safety and state awareness goal for such electric propulsion systems, enabling autonomous decision making capability for safe and efficient operation. To identify critical components in the system a Qualitative Bayesian approach using FMECA is implemented. This requires the assessment of some quantities representing the state of the electric unmanned aerial systems (e-UAS), as well as look-ahead forecasts of such states during the entire flight, presented in form of safety metrics (SM). In-service data and performance data gathered from degraded components sup- ports diagnostic and prognostic methods for these systems, but this data can be difficult to obtain as weight and packaging restrictions reduce redundancy and instrumentation on-board the vehicle. Therefore, an model-based framework should be capable or operating with limited data. In addition to data scarcity, the variability of such complex critical systems re- quires the model-based framework to reason in the presence of uncertainty, such as sensor noise, and modeling imperfections. Quantification of errors and uncertainties in the measured states and quantities is therefore a fundamental step for a precise estimation of such SMs; un-modeled uncertainty may result in erroneous state assessment and un- reliable predictions of future states of e-UAVs. Typical, centralized model-based schemes suffer from inherent disadvantages such as computational complexity, single point of failure, and scalability issues, and therefore may fail in such a complex scenario. This paper presents a methodology for developing a system level diagnostics and prognostics approach using a Qualitative Bayesian FMECA approach along with a formal uncertainty management framework for an e-UAS. In this work we demonstrate the efficacy of the framework to predict effects of sub-system level degradation on vehicle operation incorporating uncertainty management to predict future behavior under different operating conditions.

Kulkarni, Chetan↗
NASA NTRS1994

Formal results regarding metric space techniques for the study of astrophysical maps

We extend a newly developed formal system for the description of astrophyscial maps. In this formalism, we consider the difference between maps to be the distance between elements of a pseudometric space (the space of all such maps). This Ansatz allows us to measure quantitatively the difference between any two maps and to order the space of all maps. For each physical characteristic of interest, this technique assigns an 'output' function to each map; the difference between the maps is then determined from the difference between their corresponding output functions. In this present study, we show that the results of this procedure are invariant under a class of transformations of the maps and the domains of the maps. In addition, we study the propagation of errors (observational uncertainties) through this formalism. We show that the uncertainties in the output functions can be controlled provided that the signal to noise ratios in the original astrophysical maps are sufficiently high. The results of this paper thus increase the effectiveness of this formal system for the description, classification, and analysis of astrophysical maps.

Adams, Fred C.↗
NASA NTRS1982

Determining spacecraft angular position with Delta VLBI - The Voyager demonstration

A demonstration of a new VLBI-based radio metric data type for interplanetary navigation has been conducted using the Voyager spacecraft on their approach to Saturn. Pseudo-range measurements from stations on intercontinental baselines were employed, in conjunction with interferometric observations of quasars, to obtain a precise measure of the spacecraft angular position. In addition to its high accuracy this technique requires significantly less station time than conventional range and Doppler and can be used in a downlink only mode. The paper describes the data type, acquisition system, and processing procedures. Anticipated measurement accuracy is established by analysis of various system error sources. Results obtained from Voyager observations are given to show that the system performance is in accordance with expectations.

Border, J. S.↗
NASA NTRS1999

Validation of Surface Skin Temperature and Moisture Profiles Using Satellite Data

New validation techniques and metrics using satellite data have been developed to evaluate the quality of model-based estimates of surface skin temperature (Tg) and moisture profiles (q). The satellite data consist of clear sky outgoing long-wave radiation (CLR), broadband radiances from 8 to 12 mu (RadWn), brightness temperature centered around 10.8 mu (Tbb), and total precipitable water (TPW) from microwave radiometry. We show that CLR can be used to diagnose Tg. Furthermore, by using a combination of CLR and RadWn from CERES-TRMM measurements and TPW from SSM/I, we are able to identify errors in the moisture profile. Finally, three-hourly Tbb from the International Satellite Cloud Climatology Project can be used to evaluate the amplitude and diurnal variation of Tg. For purpose of illustration, Tg and q are evaluated from runs with an early version of the Goddard Earth Observing System Data Assimilation System (GEOS-2). It is found that, in general, Tg is too cold in the winter hemisphere and q is too wet in the upper atmosphere. In order to address these deficiencies, several improvements have been implemented into GEOS-2, including a Land-Surface-Model, a Moist Turbulence Scheme, and the assimilation of new TOVS retrievals. Preliminary results indicate positive impacts from each of these implementations.

Wu, Man Li C.↗
NASA NTRS1997

Object detection in natural backgrounds predicted by discrimination performance and models

Many models of visual performance predict image discriminability, the visibility of the difference between a pair of images. We compared the ability of three image discrimination models to predict the detectability of objects embedded in natural backgrounds. The three models were: a multiple channel Cortex transform model with within-channel masking; a single channel contrast sensitivity filter model; and a digital image difference metric. Each model used a Minkowski distance metric (generalized vector magnitude) to summate absolute differences between the background and object plus background images. For each model, this summation was implemented with three different exponents: 2, 4 and infinity. In addition, each combination of model and summation exponent was implemented with and without a simple contrast gain factor. The model outputs were compared to measures of object detectability obtained from 19 observers. Among the models without the contrast gain factor, the multiple channel model with a summation exponent of 4 performed best, predicting the pattern of observer d's with an RMS error of 2.3 dB. The contrast gain factor improved the predictions of all three models for all three exponents. With the factor, the best exponent was 4 for all three models, and their prediction errors were near 1 dB. These results demonstrate that image discrimination models can predict the relative detectability of objects in natural scenes.

NASA Center ARC↗
NASA NTRS2005

NASA Tech Briefs, August 2005

Topics include: Hidden Identification on Parts: Magnetic Machine-Readable Matrix Symbols; System for Processing Coded OFDM Under Doppler and Fading; Multipurpose Hyperspectral Imaging System; Magnetic-Flux-Compensated Voltage Divider; High-Performance Satellite/Terrestrial-Network Gateway; Internet-Based System for Voice Communication With the ISS; Stripline/Microstrip Transition in Multilayer Circuit Board; Dual-Band Feed for a Microwave Reflector Antenna; Quadratic Programming for Allocating Control Effort; Range Process Simulation Tool; Simulator of Space Communication Networks; Computing Q-D Relationships for Storage of Rocket Fuels; Contour Error Map Algorithm; Portfolio Analysis Tool; Glass Frit Filters for Collecting Metal Oxide Nanoparticles; Anhydrous Proton-Conducting Membranes for Fuel Cells; Portable Electron-Beam Free-Form Fabrication System; Miniature Laboratory for Detecting Sparse Biomolecules; Multicompartment Liquid-Cooling/Warming Protective Garments; Laser Metrology for an Optical-Path-Length Modulator; PCM Passive Cooling System Containing Active Subsystems; Automated Electrostatics Environmental Chamber; Estimating Aeroheating of a 3D Body Using a 2D Flow Solver; Artificial Immune System for Recognizing Patterns; Computing the Thermodynamic State of a Cryogenic Fluid; Safety and Mission Assurance Performance Metric; Magnetic Control of Concentration Gradient in Microgravity; Avionics for a Small Robotic Inspection Spacecraft; and Simulation of Dynamics of a Flexible Miniature Airplane.

Source record↗
NASA NTRS2016

Investigation of Non-Equilibrium Radiation for Earth Entry

For Earth re-entry at velocities between 8 and 11.5 km/s, the accuracy of NASA's computational uid dynamic and radiative simulations of non-equilibrium shock layer radiation is assessed through comparisons with measurements. These measurements were obtained in the NASA Ames Research Center's Electric Arc Shock Tube (EAST) facility. The experiments were aimed at measuring the spatially and spectrally resolved radiance at relevant entry conditions for both an approximate Earth atmosphere (79% N2 : 21% O2 by mole) as well as a more accurate composition featuring the trace species Ar and CO2 (78.08% N2 : 20.95% O2 : 0.04% CO2 : 0.93% Ar by mole). The experiments were configured to target a wide range of conditions, of which shots from 8 to 11.5 km/s at 0.2 Torr (26.7 Pa) are examined in this paper. The non-equilibrium component was chosen to be the focus of this study as it can account for a significant percentage of the emitted radiation for Earth re-entry, and more importantly, non-equilibrium has traditionally been assigned a large uncertainty for vehicle design. The main goals of this study are to present the shock tube data in the form of a non-equilibrium metric, evaluate the level of agreement between the experiment and simulations, identify key discrepancies and to examine critical aspects of modeling non-equilibrium radiating flows. Radiance pro les integrated over discreet wavelength regions, ranging from the Vacuum Ultra Violet (VUV) through to the Near Infra-Red (NIR), were compared in order to maximize both the spectral coverage and the number of experiments that could be used in the analysis. A previously defined non-equilibrium metric has been used to allow comparisons with several shots and reveal trends in the data. Overall, LAURA/HARA is shown to under-predict EAST by as much as 40% and over-predict by as much as 12% depending on the shock speed. DPLR/NEQAIR is shown to under-predict EAST by as much as 50% and over-predict by as much as 20% depending on the shock speed. The one standard deviation scatter in the EAST results was calculated to be 31%. An estimate for the upper bound of the absolute error in wall-directed heat flux was calculated. Below 9 km/s, where the relative difference is large, the absolute error in radiative heat flux due to non-equilibrium models is estimated to be less then 1 W/sq cm. At the highest shock speed of 11 km/s, the error in non-equilibrium is estimated to be less than 20 W/sq cm.

Shock Tube↗
NASA NTRSDate not supplied

Building Lunar Maps for Terrain Relative Navigation and Hazard Detection Applications

Terrain Relative Navigation (TRN) systems localize a spacecraft with respect to a map of the surface by comparing descent imagery to that reference map. The spacecraft position estimates can only be as accurate as the reference map itself. Accurate map products that are based on orbital reconnaissance data must be validated for navigation applications to ensure that all relevant error sources are minimized. Currently available map products have been generated for scientific applications, so the need for accurate TRN maps remains a gap to be filled for upcoming lunar lander missions, in particular missions to the South Pole region. Additionally, representative high-resolution maps that contain lander-scale features are needed for successful development and testing of Hazard Detection (HD) systems. This paper describes one of NASA’s current efforts to develop benchmark data sets that can be used for developing and testing TRN and HD algorithms as well as suggested processes and metrics for generating and validating lunar maps that can be used for navigation and hazard detection.

Lunar Maps↗
NASA NTRSDate not supplied

Planning Satellite Swarm Measurements for Climate Models: Comparing Dynamic Constraint Processing and MILP Methods

We present D-SHIELD, a challenging climate science application to plan coordinated measurements (observations) for a constellation of satellites, each containing two different sensors, each with 61 pointing angle options. The L-band and P-band radar sensors collect data fed into a soil moisture model which tracks and predicts soil moisture across 1.67 million Ground Positions (GP). Soil moisture is an important predictor of wildfires, and then a predictor of floods, landslides and debris flow after a fire. Each measurement covers multiple GP due to the sensor footprint. Each GP has a "model error" which represents the uncertainty of the the soil moisture state prediction. Model error changes at different rates for each GP as the time since last observation increases and after significant events like rain. The planner's goal is to select measurements which maximize soil moisture model improvement (reduce model uncertainty). This problem is combinatorically explosive, involving many degrees of freedom for planner choices. Good domain heuristics can find solutions within a reasonable time for our application needs but cannot be proven optimal. In this paper we compare two different planning approaches to this problem: Dynamic Constraint Processing (DCP) and Mixed Integer Linear Programming (MILP). We match inputs and metrics for both DCP and MILP algorithms to enable a direct apples-to-apples comparison. We demonstrate and discuss the trades between DCP flexibility and performance vs. MILP's promise of provable optimality.

Rich Levinson↗
NASA NTRS2022

Building Lunar Maps for Terrain Relative Navigation and Hazard Detection Applications

Terrain Relative Navigation (TRN) systems that localize a spacecraft with respect to a map of the surface by comparing descent imagery to that reference map can only be as accurate as the reference map itself. Accurate map products that are based on orbital reconnaissance data must be validated for navigation applications to ensure that all relevant error sources are minimized. Currently available map products have been generated for scientific applications, so the need for accurate TRN maps remains a gap to be filled for upcoming lunar lander missions, in particular missions to the South Pole region. Additionally, representative high-resolution maps that contain lander-scale features are needed for successful development and testing of Hazard Detection (HD) systems. This paper describes one of NASA’s current efforts to develop benchmark data sets that can be used for developing and testing TRN and HD algorithms as well as suggested processes and metrics for generating and validating lunar maps that can be used for navigation and hazard detection.

Beyer, Ross A.↗