Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Impact of GMI All-Sky Radiance Assimilation in the NASA GEOS Forecast System

The assimilation of cloud- and precipitation-affected ("all-sky") radiances has become an important focus of development at most numerical weather prediction centers. Efforts at the Global Modeling and Assimilation Office (GMAO) have focused on all-sky assimilation of GPM Microwave Imager (GMI) radiances, which became operational in the GEOS real-time production system in July 2018. Implementation of the all-sky capability required several upgrades to the GEOS hybrid 4D-EnVar assimilation infrastructure including the addition of control variables for cloud liquid, cloud ice, rain and snow, enhancements to the radiative transfer model, new hybrid background and observational error models, and modified quality control and bias correction procedures. This talk describes the impact of GMI all-sky radiance assimilation on GEOS analyses and forecasts as determined from examination of various metrics including statistics of background departures and analysis increments, forecast skill scores, and forecast sensitivity observation impact (FSOI) calculations. It is shown that in addition to the hydrometeors themselves, the initial wind, temperature and pressure fields all undergo significant dynamic adjustment in response to the analyzed cloud and precipitation features. Assimilation of GMI radiances leads to improved forecasts of lower tropospheric wind, temperature and humidity, especially in the tropics. The largest forecast improvements occur during the first 48 hours, with diminishing impact thereafter. However, combining GMI all-sky assimilation with improvements to the GEOS model physics as in the recent implementation of the real-time production system, extends these forecast improvements well in to the medium range. FSOI results based on a 24-hr moist global energy norm show that GMI radiances provide nearly uniform beneficial impact throughout the tropics, with more mixed impacts in the subtropics. While the overall impact of GMI is smaller than that of other, much more numerous microwave and hyperspectral infrared radiance types, its impact is among the largest of all radiance types on a per-observation basis.

Gelaro, Ron↗
NASA NTRS2021

Probabilistic Processing of Asynchronous Geiger-Mode Avalanche Photodiode Arrays for Background Rate Measurements

A system-level performance evaluation of Geiger-mode avalanche photodiode (GmAPD) arrays requires accurate measurement and prediction of the background rate of the device due to dark counts and other spurious detection events. Since a GmAPD detector reports only a binary value and timestamp associated with an avalanche event, dark count rates are typically measured by averaging thousands of frames to support a statistically significant measurement. For both synchronous and asynchronous detector, the Poisson distributed background rates are referenced to the time each pixel is armed. Unlike for synchronous GmAPD imagers where all the pixels are armed to an array-wide arm signal, an asynchronous pixel operates independently from its neighboring pixels; requiring the background rates to be calculated using an interarrival histogram. For both types of imagers, the background rate is typically evaluated by fitting an exponential distribution to a fixed window within a measured histogram of time intervals between detection events However, if the statistics of the background rate are insufficient – whether that is due to low population sizes, saturation, or a large dynamic range of population size across the array, the pixel, or array-wide, performance metrics may report results with varying accuracy. This paper reports on an implementation of an algorithm that evaluates GmAPD background rates based on statistical metrics rather than fixed windows. The algorithm functions by determining the appropriate integration window within the interarrival time histogram based on a per-pixel count rate set by a predetermined tolerable measurement error. The implementation of the algorithm allows us to characterize GmAPD arrays with orders of magnitude spread in background rates across the detector using common statistical parameters.

Mumolo, J.↗
NASA NTRS1978

A demonstration of dual spacecraft tracking conducted with the Viking spacecraft during the approach phase

The potential improvements in navigation capability of dual spacecraft tracking were demonstrated using Viking approach data. Under unfavorable conditions of large plasma noise, low spacecraft declination and large earth-spacecraft distance, the dual spacecraft tracking technique improved the Viking B approach accuracy based on short-arc radio metric data, by a factor of 7, to less than 200 km at Mars orbit insertion minus 3 days. From the results of an analytical expansion and the Viking demonstration with a large intentional error in Mars ephemeris, that dual spacecraft data types are shown to be insensitive to ephemeris error. Results also reveal the potential reduction of tracking time requirements during planet approach.

Chao, C. C.↗
NASA NTRS1990

Assessment of proposed fighter agility metrics

This paper presents the results of an analysis of proposed metrics to assess fighter aircraft agility. A novel framework for classifying these metrics is developed and applied. A set of transient metrics intended to quantify the axial and pitch agility of fighter aircraft is evaluated with a high fidelity, nonlinear F-18 simulation. Test techniques and data reduction method are proposed, and sensitivities to pilot introduced errors during flight testing is investigated. Results indicate that the power onset and power loss parameters are promising candidates for quantifying axial agility, while maximum pitch up and pitch down rates are for quantifying pitch agility.

Liefer, Randall K.↗
NASA NTRSDate not supplied

Space-borne Doppler Weather Radar Modeling for Radar Design Evaluation

A model has been developed to predict the reflectivity and Doppler performance of a spaceborne weather radar for atmospheric aerosol and cloud monitoring. The goal is to predict radar sensitivity, resolution, uncertainties and other key performance metrics vs. radar design parameters and hardware nonidealities. Analytical formulas are readily available to predict key performance metrics of a space-borne radar system as a function of design parameters such as antenna size, spacecraft velocity, transmit power, and receive noise figure. Effects of some system nonidealities such as antenna pointing errors, power amplifier nonlinearity and phase noise can be estimated using idealized methods. These analytical formulas use idealized forms of the antenna radiation pattern and weather statistics to predict system performance. However, it is desirable to have a more physical model based on discretized weather volumes in which the particle size distribution, Doppler distribution, and other parameters can be varied to study how the radar hardware design parameters and nonidealities affect the measurement of the weather reflectivity and Doppler characteristics. This would allow the radar designers to have more insight into what is being measured and the hardware parameters and errors that need to be carefully controlled to achieve best radar performance, as well as potential methods to calibrate the system or correct errors. This presentation will demonstrate some proof-of-concept Ka-band simulations where the discretized model agrees with the analytical formulas for a simple case where the elements in the discretized weather volume are defined by a constant reflectivity and simple velocity vectors. In particular, the system sensitivity and Doppler uncertainty are evaluated by analytical formulas as well as the discretized model. Future work will expand on this to include more complex weather scenarios and the addition of system nonidealities.

Sara Tucker↗
NASA NTRS2011

Terminal - Tactical Separation Assured Flight Environment (T-TSafe)

The Tactical Separation Assured Flight Environment (TSAFE) has been previously tested as a conflict detection and resolution tool in the en-route phase of flight. Fast time simulations of a terminal version of this tool called Terminal TSAFE (T-TSAFE) have shown promise over the current conflict detection tools. It has shown to have fewer false alerts (as low as 2 per hour) and better prediction to conflict time than Conflict Alert. The tool will be tested in the simulated terminal area of Los Angeles International Airport, in a Human-in-the-loop experiment to identify controller procedures and information requirements. The simulation will include comparisons of T-TSAFE with NASA's version of Conflict Alert. Also, some other variables such as altitude entry by the controller, which improve T-TSAFE's predictions for conflict detection, will be tested. T-TSAFE integrates features of current conflict detection tools such as Automated Terminal Proximity Alert used to alleviate compression errors in the final approach phase. Based on fast-time simulation analysis, the anticipated benefits of T-TSAFE over Conflict Alert include reduced false/missed alerts and increased time to predicted loss of separation. Other metrics that will be used to evaluate the tool's impact on the controller include controller intervention, workload, and situation awareness.

Verma, Savita Arora↗
NASA NTRSDate not supplied

WFIRST CGI: Prism and Polarizer Characterization Modes

As part of its technology demonstration, the WFIRST coronagraph instrument (CGI) will demonstrate point source spectroscopy and polarization measurements of disks. The specific implementation of spectroscopy is a zero-deviation Amici prism and slit to be placed on the planet after high contrast has been achieved by CGI. The polarization optics are a set of Wollaston prisms so that orthogonal polarization states can be measured simultaneously. The CGI spectral characterization mode, being designed and built and Goddard Space Flight Center (GSFC), has a spectral resolution of R50 and is designed to accommodate a 15% bandpass spanning 610-785nm. In order to recover Stokes information, there are two sets of Wollaston prisms clocked 45 degrees with respect to one another with each measurement taken in series. The Wollaston design and optical elements are a contribution by the Japanese Aerospace Exploration Agency, with final alignment and testing being done at GSFC. The spectroscopy mode is designed to target Methane absorption features around 730nm, keeping the spectral resolution as low as possible to improve the signal-to-noise ratio and hence reduce detection time. We highlight the requirements for these modes and address the challenge of on-orbit spectral calibration for a deployable slit in the presence of pointing drifts. Of unique interest is how the observatory error budget couples into good stellar spectrum calibration and subsequent speckle subtraction. We also provide further detail on the optomechanical design, its stability based on thermal and structural predictions, modeled performance, and operations concept. These performance metrics are simulated to demonstrate how a slit located at an arbitrary field point is homed onto the planet and converted to a calibrated spectrum.

Tyler D Groff↗
NASA NTRSDate not supplied

Strategies for Quantifying Human Space Flight Performance in the Crew Health and Performance System

The Crew Health and Performance-Probabilistic Risk Assessment (CHP-PRA) team at NASA Glenn Research Center is planning a customized approach to quantify human spaceflight performance changes with respect to changes to the CHP system functions and capabilities. Using the Directed Acyclic Graphs (DAG) initiated by NASA’s Human Systems Risk Board (HSRB) [1], the team is surveying potential candidate models and novel strategies that generate metrics suitable for supporting decision making related to how the CHP system may influence human system performance risk. One such investigation includes classic Human Reliability Analysis (HRA) models. Traditionally, HRA methods estimate the occurrence of human errors and their impact on the success of an activity when designing and operating a system. While humans perceive, interpret, decide on, and carry out a course of action, the factors affecting performance and error likelihood are commonly referred to as performance shaping factors (PSFs). Originally developed to alleviate safety concerns related to nuclear power plant operations, HRA methods such as THERP [2] and CREAM [3] dismantle an activity into tasks, requiring elemental steps to be executed, and assess their failure due to predefined PSFs. In this study, we compare generic HRA methods and those that incorporate some human spaceflight aspects, such as sleep conditions (SCREAM [4]), with respect to how they may be adopted to capture performance with an intention to mitigate detrimental outcomes elucidated by the HSRB DAGs. We suggest strategies to quantify astronaut performance specific to spaceflight activities and illustrate how such concepts may help in optimizing the CHP system capabilities with respect to Artemis missions.

dag↗
NASA NTRSDate not supplied

Enhanced Simulation Techniques in Predicting Sonic Boom Loudness Using CFD

This paper outlines advancements in predicting sonic boom loudness within the Launch, Ascent, and Vehicle Aerodynamics (LAVA) computational framework. Traditionally, a two step process consisting of a steady state computational fluid dynamics problem for near-field analysis and a far-field propagation solver for calculation of loudness metrics has been used. Improvements to this process made in this work include utilizing a high-order space marching method for mid-field computations, developing a novel output-based mesh adaptation method targeting error in near-field pressure sig-natures, and developing a robust scripting system using curvilinear grids to increase robustness and simplify the process of running large databases of simulation cases. These advancements are detailed and applied to the simulation of the X-59, presenting comparative cost and timing analyses between the prior two step workflow and the current three step procedure. We achieve increased accuracy and robustness for loudness predictions with at least a50%computational cost reduction.

CST↗
NASA NTRSDate not supplied

Near-Body Mesh Adaption for Transitional Flows Using OVERFLOW

Accurate modeling of boundary-layer transition is an important aspect of developing greener air transport technologies. In that regard, transition models based on auxiliary transport equations offer a robust approach that is easily integrated into the Reynolds-averaged Navier-Stokes (RANS) solvers. Recent workshops under NATO and AIAA have identified the verification of transport-equations-based transition modeling as a critical aspect of reducing the scatter between the predictions of different CFD codes. Follow-on work has highlighted the need for highly dense grids to achieve an asymptotic convergence of transition related flow metrics. The present work examines the role of automatic near-body mesh adaptation capability in the NASA OVERFLOW CFD solver to enable verification studies in an efficient manner, and for establishing best practices for designing grids for the RANS-based transition models. A sensor function relevant to the Langtry-Menter \gamma-Re_{\theta t}\ transition model has been identified and used for error-based mesh adaptation for canonical configurations comprising a flat plate, and the S809 and NLR-7301 airfoils. The efficacy of the mesh adaptation approach is assessed for flow conditions involving multiple transition scenarios such as natural transition, separation-induced transition, and shock-induced transition. The results from this exploratory study indicate that the meshes adapted using the proposed sensor provide solutions that approach the references solutions obtained with uniformly refined hand-crafted meshes, in terms of the chosen metrics, and yield modest yet significant savings in grid count. We also highlight areas for improvement in the grid adaptation methodology within OVERFLOW.

CFD↗
NASA NTRSDate not supplied

Near-Body Mesh Adaption for Transitional Flows Using OVERFLOW

Accurate modeling of boundary-layer transition is an important aspect of developing greener air transport technologies. In that regard, transition models based on auxiliary transport equations offer a robust approach that is easily integrated into the Reynolds-averaged Navier-Stokes (RANS) solvers. Recent workshops under NATO and AIAA have identified the verification of transport-equations-based transition modeling as a critical aspect of reducing the scatter between the predictions of different CFD codes. Follow-on work has highlighted the need for highly dense grids to achieve an asymptotic convergence of transition related flow metrics. The present work examines the role of automatic near-body mesh adaptation capability in the NASA OVERFLOW CFD solver to enable verification studies in an efficient manner, and for establishing best practices for designing grids for the RANS-based transition models. A sensor function relevant to the Langtry-Menter \gamma-Re_{\theta t}\ transition model has been identified and used for error-based mesh adaptation for canonical configurations comprising a flat plate, and the S809 and NLR-7301 airfoils. The efficacy of the mesh adaptation approach is assessed for flow conditions involving multiple transition scenarios such as natural transition, separation-induced transition, and shock-induced transition. The results from this exploratory study indicate that the meshes adapted using the proposed sensor provide solutions that approach the references solutions obtained with uniformly refined hand-crafted meshes, in terms of the chosen metrics, and yield modest yet significant savings in grid count. We also highlight areas for improvement in the grid adaptation methodology within OVERFLOW.

CFD↗
NASA NTRS2017

Comparison of Commonly-Used Microwave Radiative Transfer Models for Snow Remote Sensing

This paper reviews four commonly-used microwave radiative transfer models that take different electromagnetic approaches to simulate snow brightness temperature (T(sub B)): the Dense Media Radiative Transfer - Multi-Layer model (DMRT-ML), the Dense Media Radiative Transfer - Quasi-Crystalline Approximation Mie scattering of Sticky spheres (DMRT-QMS), the Helsinki University of Technology n-Layers model (HUT-nlayers) and the Microwave Emission Model of Layered Snowpacks (MEMLS). Using the same extensively measured physical snowpack properties, we compared the simulated T(sub B) at 11, 19 and 37 GHz from these four models. The analysis focuses on the impact of using different types of measured snow microstructure metrics in the simulations. In addition to density, snow microstructure is defined for each snow layer by grain optical diameter (Do) and stickiness for DMRT-ML and DMRT-QMS, mean grain geometrical maximum extent (D(sub max)) for HUT n-layers and the exponential correlation length for MEMLS. These metrics were derived from either in-situ measurements of snow specific surface area (SSA) or macrophotos of grain sizes (D(sub max)), assuming non-sticky spheres for the DMRT models. Simulated T(sub B) sensitivity analysis using the same inputs shows relatively consistent T(sub B) behavior as a function of Do and density variations for the vertical polarization (maximum deviation of 18 K and 27 K, respectively), while some divergences appear in simulated variations for the polarization ratio (PR). Comparisons with ground based radiometric measurements show that the simulations based on snow SSA measurements have to be scaled with a model-specific factor of Do in order to minimize the root mean square error (RMSE) between measured and simulated T(sub B). Results using in-situ grain size measurements (SSA or D(sub max), depending on the model) give a mean T(sub B) RMSE (19 and 37 GHz) of the order of 16-26 K, which is similar for all models when the snow microstructure metrics are scaled. However, the MEMLS model converges to better results when driven by the correlation length estimated from in-situ SSA measurements rather than D(sub max) measurements. On a practical level, this paper shows that the SSA parameter, a snow property that is easy to retrieve in-situ, appears to be the most relevant parameter for characterizing snow microstructure, despite the need for a scaling factor.

Snow↗
NASA NTRS1993

Methods of evaluating the effects of coding on SAR data

It is recognized that mean square error (MSE) is not a sufficient criterion for determining the acceptability of an image reconstructed from data that has been compressed and decompressed using an encoding algorithm. In the case of Synthetic Aperture Radar (SAR) data, it is also deemed to be insufficient to display the reconstructed image (and perhaps error image) alongside the original and make a (subjective) judgment as to the quality of the reconstructed data. In this paper we suggest a number of additional evaluation criteria which we feel should be included as evaluation metrics in SAR data encoding experiments. These criteria have been specifically chosen to provide a means of ensuring that the important information in the SAR data is preserved. The paper also presents the results of an investigation into the effects of coding on SAR data fidelity when the coding is applied in (1) the signal data domain, and (2) the image domain. An analysis of the results highlights the shortcomings of the MSE criterion, and shows which of the suggested additional criterion have been found to be most important.

Dutkiewicz, Melanie↗
NASA NTRS2023

A Blind Convolutional Deep Autoencoder for Spectral Unmixing of Hyperspectral Images Over Waterbodies

Harmful algal blooms have dangerous repercussions for biodiversity, the ecosystem, and public health. Automatic identification based on remote sensing hyperspectral image analysis provides a valuable mechanism for extracting the spectral signatures of harmful algal blooms and their respective percentage in a region of interest. This paper proposes a new model called a non-symmetrical autoencoder for spectral unmixing to perform endmember extraction and fractional abundance estimation. The model is assessed in benchmark datasets, such as Jasper Ridge and Samson. Additionally, a case study of the HSI2 image acquired by NASA over Lake Erie in 2017 is conducted for extracting optical water types. The results using the proposed model for the benchmark datasets improve unmixing performance, as indicated by the spectral angle distance compared to five baseline algorithms. Improved results were obtained for various metrics. In the Samson dataset, the proposed model outperformed other methods for water (0.060) and soil (0.025) endmember extraction. Moreover, the proposed method exhibited superior performance in terms of mean spectral angle distance compared to the other five baseline algorithms. The non-symmetrical autoencoder for the spectral unmixing approach achieved better results for abundance map estimation, with a root mean square error of 0.091 for water and 0.187 for soil, compared to the ground truth. For the Jasper Ridge dataset, the non-symmetrical autoencoder for the spectral unmixing model excelled in the tree (0.039) and road (0.068) endmember extraction and also demonstrated improved results for water abundance maps (0.1121). The proposed model can identify the presence of chlorophyll-a in waterbodies. Chlorophyll-a is an essential indicator of the presence of the different concentrations of macrophytes and cyanobacteria. The non-symmetrical autoencoder for spectral unmixing achieves a value of 0.307 for the spectral angle distance metric compared to a reference ground truth spectral signature of chlorophyll-a. The source code for the proposed model, as implemented in this manuscript, can be found at https://github.com/EstefaniaAlfaro/autoencoder_owt_spectral.git.

hyperspectral imaging↗
NASA NTRS2015

Ocean Heat and Carbon Uptake in Transient Climate Change: Identifying Model Uncertainty

Global warming on decadal and centennial timescales is mediated and ameliorated by the oceansequestering heat and carbon into its interior. Transient climate change is a function of the efficiency by whichanthropogenic heat and carbon are transported away from the surface into the ocean interior (Hansen et al. 1985).Gregory and Mitchell (1997) and Raper et al. (2002) were the first to identify the importance of the ocean heat uptakeefficiency in transient climate change. Observational estimates (Schwartz 2012) and inferences from coupledatmosphere-ocean general circulation models (AOGCMs; Gregory and Forster 2008; Marotzke et al. 2015), suggest thatocean heat uptake efficiency on decadal timescales lies in the range 0.5-1.5 W/sq m/K and is thus comparable to theclimate feedback parameter (Murphy et al. 2009). Moreover, the ocean not only plays a key role in setting the timing ofwarming but also its regional patterns (Marshall et al. 2014), which is crucial to our understanding of regional climate,carbon and heat uptake, and sea-level change. This short communication is based on a presentation given by A.Romanou at a recent workshop, Oceans Carbon and Heat Uptake: Uncertainties and Metrics, co-hosted by US CLIVARand OCB. As briefly reviewed below, we have incomplete but growing knowledge of how ocean models used in climatechange projections sequester heat and carbon into the interior. To understand and thence reduce errors and biases inthe ocean component of coupled models, as well as elucidate the key mechanisms at work, in the final section we outlinea proposed model intercomparison project named FAFMIP. In FAFMIP, coupled integrations would be carried out withprescribed overrides of wind stress and freshwater and heat fluxes acting at the sea surface.

periodic variations↗
NASA NTRS2022

Validation of IMERG Oceanic Precipitation over Kwajalein

The integrated Multi-satellitE Retrievals for GPM (IMERG) Version V05B and V06B precipitation products from the Global Precipitation Measurement (GPM) mission are validated against ground-based observations from the Kwajalein Polarimetric S-band Weather Radar (KPOL) deployed at Kwajalein Atoll in the central Pacific Ocean. Such a validation is particularly important as comprehensive surface measurements over the oceans are practically infeasible, which hampers the identification of possible errors, and improvement of future versions of IMERG and other satellitebased retrieval algorithms. The V05B and V06B IMERG products are validated at their native 0.1°, 30 min resolution from 2014 to 2018 based on both volumetric and categorical metrics. This validation study indicates that precipitation rates from both IMERG V05B and V06B are underestimated with respect to radar surface estimates, but the underestimation is much reduced from V05B IMERG V06B outperforms V05B with reduced systematic bias and improved precipitation detectability. The IMERG performance is further traced back to its individual sensors and morphing-based algorithms. The overall underestimation in V05B is mainly driven by the negative relative biases from morphing-based algorithms which are largely corrected in V06B. Imagers perform generally better than sounders because of the usage of low-frequency channels in imagers which can better detect emission signals by the hydrometeors. Among imagers, the GPM Microwave Imager (GMI) and Advanced Microwave Scanning Radiometer Version 2 (AMSR2) are the best, followed by Special Sensor Microwave Imager/Sounder (SSMIS). Among sounders, the Microwave Humidity Sounder (MHS) is the best, followed by Advanced Technology Microwave Sounder (ATMS) and the Sounder for Atmospheric Profiling of Humidity in the Intertropics by Radiometry (SAPHIR) for V06B. Among all categories, morph-only and IR+morph only perform better than SAPHIR. SAPHIR shows the worst performance among all categories, likely due to its limited channel selection. It is envisaged that these results will improve our understanding of IMERG performance over oceans and aid in the improvement of future versions of IMERG.

KPOL↗
NASA NTRS2011

Reliability Analysis and Standardization of Spacecraft Command Generation Processes

center dot In order to reduce commanding errors that are caused by humans, we create an approach and corresponding artifacts for standardizing the command generation process and conducting risk management during the design and assurance of such processes. center dot The literature review conducted during the standardization process revealed that very few atomic level human activities are associated with even a broad set of missions. center dot Applicable human reliability metrics for performing these atomic level tasks are available. center dot The process for building a "Periodic Table" of Command and Control Functions as well as Probabilistic Risk Assessment (PRA) models is demonstrated. center dot The PRA models are executed using data from human reliability data banks. center dot The Periodic Table is related to the PRA models via Fault Links.

human errors.↗
NASA NTRS1980

Measuring the development process: A tool for software design evaluation

The design metrics evaluator (DME), a component of an automated software design analysis system, is described. The DME quantitatively evaluates software design attributes. Its use directs attention to areas of a procedure, module, or complete program having a high potential for error.

Moy, S. S.↗