Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Using OSSEs to Evaluate the Impacts of Geostationary Infrared Sounders

An observing system simulation experiment (OSSE) was performed to assess the impact of assimilating hyperspectral infrared (IR) radiances from geostationary orbit on numerical weather prediction, with a focus on the proposed sounder on board the Geostationary Extended Observations (GeoXO) program’s central satellite. Infrared sounders on a geostationary platform would fill several gaps left by IR sounders on polar-orbiting satellites, and the increased temporal resolution would allow the observation of weather phenomena evolution. The framework for this OSSE was the Global Modeling and Assimilation Office (GMAO) OSSE system, which includes a full suite of meteorological observations. The experiment additionally assimilated four identical IR sounders from geostationary orbit to create a “ring” of vertical profiling observations. Based on the experimentation, assimilation of the IR sounders provided a beneficial impact on the analyzed mass and wind fields, particularly in the tropics, and produced an error reduction in the initial 24–48 h of the subsequent forecasts. Specific attention was paid to the impact of the GeoXO Sounder (GXS) over the contiguous United States (CONUS) as this is a region that is well-observed and as such difficult to improve. The forecast sensitivity to observation impact (FSOI) metric, computed across all four synoptic times over the CONUS, reveals that the GXS had the largest impact on the 24-h forecast error of the assimilated hyperspectral infrared satellite radiances as measured using a moist energy error norm. Based on this analysis, the proposed GXS has the potential to improve numerical weather prediction globally and over the CONUS.

OSSE↗
NASA NTRS2016

Regime-Based Evaluation of Cloudiness in CMIP5 Models

The concept of Cloud Regimes (CRs) is used to develop a framework for evaluating the cloudiness of 12 fifth Coupled Model Intercomparison Project (CMIP5) models. Reference CRs come from existing global International Satellite Cloud Climatology Project (ISCCP) weather states. The evaluation is made possible by the implementation in several CMIP5 models of the ISCCP simulator generating for each gridcell daily joint histograms of cloud optical thickness and cloud top pressure. Model performance is assessed with several metrics such as CR global cloud fraction (CF), CR relative frequency of occurrence (RFO), their product (long-term average total cloud amount [TCA]), cross-correlations of CR RFO maps, and a metric of resemblance between model and ISCCP CRs. In terms of CR global RFO, arguably the most fundamental metric, the models perform unsatisfactorily overall, except for CRs representing thick storm clouds. Because model CR CF is internally constrained by our method, RFO discrepancies yield also substantial TCA errors. Our findings support previous studies showing that CMIP5 models underestimate cloudiness. The multi-model mean performs well in matching observed RFO maps for many CRs, but is not the best for this or other metrics. When overall performance across all CRs is assessed, some models, despite their shortcomings, apparently outperform Moderate Resolution Imaging Spectroradiometer (MODIS) cloud observations evaluated against ISCCP as if they were another model output. Lastly, cloud simulation performance is contrasted with each model's equilibrium climate sensitivity (ECS) in order to gain insight on whether good cloud simulation pairs with particular values of this parameter.

cloud climatology↗
NASA NTRS2018

Assessment and Mitigation of the Effects of Noise on Habitability in Deep Space Environments: Report on Non-Auditory Effects of Noise

This presentation reviews non-auditory effects of noise relevant to habitable volume requirements in cislunar space. The non-auditory effects of noise in future long-term space habitats are likely to be impactful on team and individual performance, sleep, and cognitive well-being. Recommendations are provided for future standards and procedures for long-term space flight habitats, along with recommendations for NASA's Human Research Program in support of DST mission success. The following non-auditory effects of noise are addressed: performance-teamwork; sleep disturbance; and cognitive well-being, all of which impact crew performance and readiness to perform. It reviews the relevance of research performed on earth to the space environment, the quality of the measurement metrics used, and potentially significant research gaps. Auditory effects of noise are considered to the degree that measurement metrics overlap with non-auditory effects.

Readback Errors↗
NASA NTRS2001

Integrated Modeling for the Next Generation Space Telescope "Yardstick" Concept

The so-called NASA "Yardstick" design concept for the Next Generation Space Telescope presents unique challenges for systems-level analysis. Simulations that integrate controls, optics, thermal, and structural models are required to evaluate baseline performance, study design sensitivities, and perform design optimization. An integrated modeling approach was chosen using a combination of commercial off-the-shelf and "in-house" developed codes. The resulting capability provides a foundation for linear and non-linear analysis, using both the time and frequency-domain methods. It readily allows various combinations of design parameters and environmental loads to be evaluated directly in terms of key science-related metrics, in this case the scalar RMS (root mean square) line-of-sight and RMS wavefront errors. This presentation first addresses the development of the component, or discipline, models for the Yardstick design. It will then proceed to present the integration of the component models, using linear-systems approaches, in order to support two of the most critical baseline performance analyses: jitter and thermal-elastic stability of the optical telescope assembly (OTA). The results of the jitter analysis indicate that disturbances from the reaction wheels coupled with the lightly-damped and highly-flexible structure present significant challenges to the baseline line-of-sight control architecture. Vibration isolation will be required to meet jitter error requirements. The results of the thermal-elastic analysis indicate that the mirror segment displacements due to ground-to-orbit cool-down of the telescope are within the expected capture range of the segment rigid-body control actuators. This means we will be able to align and phase the primary mirror. However, the results for the analysis of the thermal transient response following an attitude maneuver (slew) show that this telescope design is not sufficiently stable, passively, to meet the wavefront error requirements. Structural re-design is one possibility; alternatively, active thermal control of the OTA may be considered. The Yardstick integrated models were successfully used to demonstrate the feasibility of two thermal control strategies.

Mosier, Gary E.↗
NASA NTRS2003

Probabilistic Causal Analysis for System Safety Risk Assessments in Commercial Air Transport

Aviation is one of the critical modes of our national transportation system. As such, it is essential that new technologies be continually developed to ensure that a safe mode of transportation becomes even safer in the future. The NASA Aviation Safety Program (AvSP) is managing the development of new technologies and interventions aimed at reducing the fatal aviation accident rate by a factor of 5 by year 2007 and by a factor of 10 by year 2022. A portfolio assessment is currently being conducted to determine the projected impact that the new technologies and/or interventions may have on reducing aviation safety system risk. This paper reports on advanced risk analytics that combine the use of a human error taxonomy, probabilistic Bayesian Belief Networks, and case-based scenarios to assess a relative risk intensity metric. A sample case is used for illustrative purposes.

Luxhoj, James T.↗
NASA NTRS2019

Calibration and Data Analysis Recommendations for Three-Component Moment Balances

Fundamental characteristics of design, calibration, and application of three-component moment balances are investigated in great detail. These balances are typically used to determine loads on control surfaces, canards, or other parts that are attached to a wind tunnel model. First, three different descriptions of the load state of a moment balance are reviewed. Then, load transformations between different load formats and the combined load diagram for two of the three load components are discussed. An error analysis showed that it is critical to maximize the product of the distance between the bending moment gages and their sensitivities in order to minimize the overall error in the normal force prediction. In addition, it is important to apply a sufficient number of calibration loadings near the first bending moment gage. Then, unwanted near-linear dependencies between the two bending moment gage outputs can be avoided. The error in the bending moment prediction is also investigated that results from the elastic deformation of the metric part of the balance under load. Finally, the application of the Non-Iterative Method to three-component moment balance calibration data is described in order to obtain regression models that can be used to predict loads from measured outputs during a wind tunnel test.

Strain-Gage Balance↗
NASA NTRS2021

Analysis of Mars 2020 Perseverance Entry, Descent, and Landing Attitude Initialization Performance

The Mars 2020 Perseverance rover successfully landed at Jezero Crater on February 18, 2021. To perform this feat, the entry, descent, and landing navigation filter must be initialized with an estimate of the spacecraft state, which includes attitude from the cruise attitude control subsystem. Accuracy of this initial attitude estimate has an impact on important metrics such as touchdown velocity. This paper presents a post-landing reconstruction of the attitude initialization error budget, showing that the requirement of 0.15 deg ($3\sigma$, per axis) was met. Each error source is described and analyzed, including flight telemetry where possible. Analysis of the error budget shows that it is driven by systematic sun sensor errors, star-scanner-to-sun-sensor alignment stability, and inertial-measurement-unit-to-sun-sensor alignment stability. Finally, a contingency plan to initialize the navigation filter in the event of a star scanner failure is presented. While this plan was not needed in flight, results indicate that the attitude initialization requirement could have still been met in this off-nominal scenario.

Lo, Kevin D.↗
NASA NTRS1982

A study of image quality for radar image processing

Methods developed for image quality metrics are reviewed with focus on basic interpretation or recognition elements including: tone or color; shape; pattern; size; shadow; texture; site; association or context; and resolution. Seven metrics are believed to show promise as a way of characterizing the quality of an image: (1) the dynamic range of intensities in the displayed image; (2) the system signal-to-noise ratio; (3) the system spatial bandwidth or bandpass; (4) the system resolution or acutance; (5) the normalized-mean-square-error as a measure of geometric fidelity; (6) the perceptual mean square error; and (7) the radar threshold quality factor. Selective levels of degradation are being applied to simulated synthetic radar images to test the validity of these metrics.

King, R. W.↗
NASA NTRS2020

Observing System Simulation Experiments Investigating Atmospheric Motion Vectors and Radiances from a Constellation of 4-5 μm Infrared Sounders

A set of Observing System Simulation Experiments (OSSEs) was performed to investigate the utility of a constellation of passive infrared spectrometers, strategically designed with the aim of deriving the three-dimensional retrievals of the horizontal wind via atmospheric motion vectors (AMVs) from instruments with the spectral resolution of an infrared sounder. The instrument and constellation designs were performed in the context of the Midwave Infrared Sounding of Temperature and humidity in a Constellation for Winds, or MISTiC Winds. The Global Modeling and Assimilation Office OSSE system, which includes a full suite of operational meteorological observations, served as the control. To illustrate the potential impact of this observing strategy, two experiments were performed by adding the new simulated observations to the control. First, perfect (error-free) simulated AMVs and radiances were assimilated. Second, the data were made imperfect by adding realistic modeled errors to the AMVs and radiances that were assimilated. The experimentation showed beneficial impacts on both the mass and wind fields, as based on analysis verification, forecast verification, and the assessment of the observations using the Forecast Sensitivity to Observation Impact (FSOI) metric. In all variables and metrics, the impacts of the imperfect observations were smaller than those of the perfect observations, though much of the positive benefit was retained. The FSOI metric illustrated two key points. First, the largest impacts were seen in the middle troposphere AMVs, which is a targeted capability of the constellation strategy. Second, the addition of modeled errors showed that the assimilation system was unable to fully exploit the 4.3 μm carbon dioxide absorption radiances.

Will McCarty↗
NASA NTRS2021

Observing System Simulation Experiments Investigating Atmospheric Motion Vectors and Radiances from a Constellation of 4–5-μm Infrared Sounders

A set of observing system simulation experiments (OSSEs) was performed to investigate the utility of a constellation of passive infrared spectrometers, strategically designed with the aim of deriving the three-dimensional retrievals of the horizontal wind via atmospheric motion vectors (AMVs) from instruments with the spectral resolution of an infrared sounder. The instrument and constellation designs were performed in the context of the Midwave Infrared Sounding of Temperature and humidity in a Constellation for Winds (MISTiC Winds). The Global Modeling and Assimilation Office OSSE system, which includes a full suite of operational meteorological observations, served as the control. To illustrate the potential impact of this observing strategy, two experiments were performed by adding the new simulated observations to the control. First, perfect (error free) simulated AMVs and radiances were assimilated. Second, the data were made imperfect by adding realistic modeled errors to the AMVs and radiances that were assimilated. The experimentation showed beneficial impacts on both the mass and wind fields, as based on analysis verification, forecast verification, and the assessment of the observations using the forecast sensitivity to observation impact (FSOI) metric. In all variables and metrics, the impacts of the imperfect observations were smaller than those of the perfect observations, although much of the positive benefit was retained. The FSOI metric illustrated two key points. First, the largest impacts were seen in the middle troposphere AMVs, which is a targeted capability of the constellation strategy. Second, the addition of modeled errors showed that the assimilation system was unable to fully exploit the 4.3-μm carbon dioxide absorption radiances.

Will McCarty↗
NASA NTRS1974

Upper bounds on sequential decoding performance parameters

This paper presents the best obtainable random coding and expurgated upper bounds on the probabilities of undetectable error, of t-order failure (advance to depth t into an incorrect subset), and of likelihood rise in the incorrect subset, applicable to sequential decoding when the metric bias G is arbitrary. Upper bounds on the Pareto exponent are also presented. The G-values optimizing each of the parameters of interest are determined, and are shown to lie in intervals that in general have nonzero widths. The G-optimal expurgated bound on undetectable error is shown to agree with that for maximum likelihood decoding of convolutional codes, and that on failure agrees with the block code expurgated bound. Included are curves evaluating the bounds for interesting choices of G and SNR for a binary-input quantized-output Gaussian additive noise channel.

Jelinek, F.↗
NASA NTRS1995

Orbit determination of highly elliptical Earth orbiters using improved Doppler data-processing modes

A navigation error covariance analysis of four highly elliptical Earth orbits is described, with apogee heights ranging from 20,000 to 76,800 km and perigee heights ranging from 1,000 to 5,000 km. This analysis differs from earlier studies in that improved navigation data-processing modes were used to reduce the radio metric data. For this study, X-band (8.4-GHz) Doppler data were assumed to be acquired from two Deep Space Network radio antennas and reconstructed orbit errors propagated over a single day. Doppler measurements were formulated as total-count phase measurements and compared to the traditional formulation of differenced-count frequency measurements. In addition, an enhanced data-filtering strategy was used, which treated the principal ground system calibration errors affecting the data as filter parameters. Results suggest that a 40- to 60-percent accuracy improvement may be achievable over traditional data-processing modes in reconstructed orbit errors, with a substantial reduction in reconstructed velocity errors at perigee. Historically, this has been a regime in which stringent navigation requirements have been difficult to meet by conventional methods.

Estefan, J. A.↗
NASA NTRS1985

Strategies for FH/MFSK signaling with diversity in worst-case partial-band noise

Optimum diversity and worst-case partial-band noise jamming conditions have been derived for noncoherent energy detection of frequency-hopped (FH) M-ary frequency-shift keyed (MFSK) signals using a soft-chip decision suboptimum linear combining metric with perfect jamming-state side information. However, the assumption implicit in previous publications is that the error is first maximized over the jammer's partial-band duty factor for arbitrary diversity, and the result is then minimized over the amount of diversity. This paper shows that if the order of optimization is reversed, different conditions and performance are produced; that is, the previous solution is not a saddlepoint. This introduces some game-theoretic considerations for the communicator and the jammer, the risks and advantages of which are explored.

Levitt, B. K.↗
NASA NTRS1994

Enhanced orbit determination filter: Inclusion of ground system errors as filter parameters

The theoretical aspects of an orbit determination filter that incorporates ground-system error sources as model parameters for use in interplanetary navigation are presented in this article. This filter, which is derived from sequential filtering theory, allows a systematic treatment of errors in calibrations of transmission media, station locations, and earth orientation models associated with ground-based radio metric data, in addition to the modeling of the spacecraft dynamics. The discussion includes a mathematical description of the filter and an analytical comparison of its characteristics with more traditional filtering techniques used in this application. The analysis in this article shows that this filter has the potential to generate navigation products of substantially greater accuracy than more traditional filtering procedures.

Masters, W. C.↗
NASA NTRS2014

Strain Gage Load Calibration of the Wing Interface Fittings for the Adaptive Compliant Trailing Edge Flap Flight Test

The safety-of-flight parameters for the Adaptive Compliant Trailing Edge (ACTE) flap experiment require that flap-to-wing interface loads be sensed and monitored in real time to ensure that the structural load limits of the wing are not exceeded. This paper discusses the strain gage load calibration testing and load equation derivation methodology for the ACTE interface fittings. Both the left and right wing flap interfaces were monitored; each contained four uniquely designed and instrumented flap interface fittings. The interface hardware design and instrumentation layout are discussed. Twenty-one applied test load cases were developed using the predicted in-flight loads. Pre-test predictions of strain gage responses were produced using finite element method models of the interface fittings. Predicted and measured test strains are presented. A load testing rig and three hydraulic jacks were used to apply combinations of shear, bending, and axial loads to the interface fittings. Hardware deflections under load were measured using photogrammetry and transducers. Due to deflections in the interface fitting hardware and test rig, finite element model techniques were used to calculate the reaction loads throughout the applied load range, taking into account the elastically-deformed geometry. The primary load equations were selected based on multiple calibration metrics. An independent set of validation cases was used to validate each derived equation. The 2-sigma residual errors for the shear loads were less than eight percent of the full-scale calibration load; the 2-sigma residual errors for the bending moment loads were less than three percent of the full-scale calibration load. The derived load equations for shear, bending, and axial loads are presented, with the calculated errors for both the calibration cases and the independent validation load cases.

Miller, Eric J.↗
NASA NTRSDate not supplied

Error Localization Examples: Looking for a Needle in a Hay-stack

Finite element models (FEM) are routinely developed and used during fabrication of high dollar-value hardware. NASA as part of the pre-flight certification of launch vehicles routinely conducts vibration and static tests to calibrate models used for flight-risk assessments. As part of the calibration process, certain areas in the model are modified, using engineering judgment and sensitivity analysis, to match the test results. Unfortunately, tools to identify problem areas in the FEM using test data directly are scarce. Over the years, Error Localization Algorithms (ELA) have been proposed with very limited success. Recently, the Analytical Dynamics Model Improvement (ADMI) algorithm, which computes closed-form mass and stiffness corrections to match the test data exactly, have been shown to be effective for error localization. The paper will present several FEM example problems where ELA is used with simulated test data to determine FEM problem areas. For each example, the correct answer is shown along with ELA results. It is shown that the ELA process is able to identify general problem areas in the FEM, which are consistent with known model perturbations. However, in most cases the ELA identified area of improvement is larger than the true answer. Nonetheless, with proper optimization tools, calibration results using the ELA identified areas provide excellent results.

error localization↗
NASA NTRS2022

Robustness of Vegetation Optical Depth Retrievals Based on L-Band Global Radiometry

Microwave vegetation optical depth (VOD) and soil moisture (SM) can be simultaneously retrieved based on L-band radiometry with polarization information. VOD is indicative of the vegetation water content (VWC) because it captures the extinction of land surface emission. If the connectivity of VOD to VWC is robust, the pair of VWC-SM observations can be viable bases for understanding soil–plant–atmosphere water relations, providing new perspectives on ecosystem science. Simultaneous SM–VOD retrievals are feasible by inverting the τ−ω model with two independent datasets in dual-channel algorithms. However, given correlated satellite vertical and horizontal brightness temperatures (TBs; TB v and TB h ), an ill-posed inverse problem arises where TB errors result in high uncertainties of retrievals. In this study, we apply the degrees-of-information (DoI) metric and propose a signal-to-noise ratio (SNR) metric to assess the “retrievability” of VOD given the Soil Moisture Active Passive (SMAP) TB v –TB h linear dependence. The application of these metrics allows determining where the VOD retrievals are robust and reliable. This is a necessary step in supporting the applications of VOD in ecology and hydrology. Results show that regions with mainly nonwoody vegetation have the best potential for VOD retrievals, though regularization is necessary. We then assess VOD time variations from two regularization products that reduce the impact of underdetermined inversions: the L3 dual-channel algorithm (L3-DCA) and the multitemporal dual-channel algorithm (MTDCA), which constrain VOD time dynamics with and without using a priori VOD climatology, respectively. Though they both reduce noise, especially in the VOD retrievals, they result in differences in VOD seasonal amplitude and coupling to SM at high frequencies as we outline here.

Microwave↗