Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

Adjoint Estimation of Observation Impact Explored with an Observing System Simulation Experiment

In an Observing System Simulation Experiment, the full, true state of the simulated atmosphere is known. This knowledge allows the direct calculation of analysis and forecast errors, and also may be used in conjunction with an adjoint tool to calculate metrics that are unachievable in the real world. For example, the errors that result from the use of the analysis state as verification when running adjoint calculations of observation impact on a forecast can be quantified. The adjoint itself can also be used to operate on the analysis state rather than on the forecast state. Results from these and other adjoint experiments using the NASA/GMAO OSSE framework will be presented.

Prive, Nikki↗
NASA NTRS2016

Investigation of Optimal Digital Image Correlation Patterns for Deformation Measurement

Digital image correlation (DIC) relies on the surface texture of a specimen to measure deformation. When the specimen itself has little or no texture, a pattern is applied to the surface which deforms with the specimen and acts as an artificial surface texture. Because the applied pattern has an effect on the accuracy of DIC, an ideal pattern is sought for which the error introduced into DIC measurements is minimal. In this work, a study is performed in which several DIC pattern quality metrics from the literature are correlated to DIC measurement error. The resulting correlations give insight on the optimality of DIC patterns in general. Optimizations are then performed to produce patterns which are well suited for DIC. These patterns are tested to show their relative benefits. Chief among these benefits are a reduction in error of approximately 30 with respect to a randomly generated pattern.

Bomarito, G. F.↗
NASA NTRS2019

Adjoint-Based Anisotropic Mesh Adaptation for a Stabilized Finite-Element Flow Solver

An adjoint solver is implemented in the FUN3D stabilized finite-element flow solver. The adjoint solution is used to generate anisotropic, adapted meshes to control error in scalar out- put functionals, such as lift or drag coefficient. The inviscid and turbulent adjoints are verified with a finite-difference derivative approximation and can be used in design optimization in addition to mesh adaptation. The adjoint capability represents an extension of previous research using the FUN3D stabilized finite-element capability with metric-based mesh adaptation and interpolation-based error estimates to generate highly anisotropic adapted meshes for turbulent flows. In the present work, a metric-based approach is again utilized, where the adjoint and the primal solutions both contribute to the generation of a metric tensor field that is subsequently used to produce the required anisotropic mesh for each adaptation cycle. Adaptive results are then shown for an inviscid supersonic flow over a diamond airfoil, inviscid transonic flow over the ONERA M6 wing, and viscous laminar flow over the NACA 0012 airfoil, all using drag force as the output functional. Adjoint-based adaptation is compared with a multiscale solution-based approach that controls the L norm of Mach number interpolation error to demonstrate the effectiveness and effciency of the adjoint-based adaptive mesh technology.

Balan, Aravind↗
NASA NTRS2004

Improvement of the TOPEX and Jason Orbit Time Series: Precision Orbit Determination, Calibration, Validation and Improvement Through the Combined Reduction and Analysis of GPS, SLR, DORIS and Altimeter Data

Orbit error is a major component in the overall error budget of all altimeter satellite missions. Jason-I is no exception and a 1 cm radial orbit accuracy goal has been set, which represents a factor of two improvement over what is currently being achieved for TOPEX/Poseidon (TP). Our current analysis suggests this goal has been met and even improved upon, but the challenge is to be able to continually achieve this high accuracy, verify the performance and characterize and quantify the remaining errors over the lifetime of the mission. The computation, verification and error characterization of such high accuracy orbits requires the reduction and analysis of all available tracking data (GPS, SLR, DORIS and altimeter). Current analysis also indicates the history of TP orbits can be further improved employing new solution strategies developed and tested on Jason-I. Our research focuses on the calibration, validation and improvement of orbit accuracies using all available tracking data including altimetry. We will compute and distribute well centered Jason orbits with an accuracy of better than 1-cm in the radial component. In addition to the orbits themselves, a characterization of the orbit error will be distributed and accumulated as a time series of orbit performance metrics to track anomalies and trends. The long time series of orbit error characterization will enable a better understanding of the remaining orbit errors and its impact on the altimeter data analysis. As part of this research effort we are also significantly improving the current level of TP orbit accuracy, re-computing new high-accuracy TP orbits from the beginning of the TP mission and continuing into the future (as long as TP is healthy). Our funded research effort will result in a complete and consistent time series of improved orbits for both TP and Jason, significantly benefiting the long time series of altimeter data analysis and the TP/Jason dual mission. The resultant high accuracy orbits and the characterization of their error will allow further improvements to the accuracy and overall quality of the altimeter measurement time series making possible further strides in radar altimeter remote sensing.

Luthcke, Scott B.↗
DOE OSTI2021

A spatial evaluation of Arctic sea ice and regional limitations in CMIP6 historical simulations

The Arctic sea ice response to a warming climate is assessed in a subset of models participating in Phase 6 of the Coupled Model Intercomparison Project (CMIP6), using several metrics in comparison with satellite observations and results from the Pan-Arctic Ice Ocean Modeling and Assimilation System and the Regional Arctic System Model. Our study examines the historical representation of sea ice extent, volume, and thickness using spatial analysis metrics, such as the integrated ice-edge error, Brier score, and Spatial Probability Score. We find that the CMIP6 multi-model mean captures the mean annual cycle and 1979-2014 sea ice trends remarkably well. However, individual models experience a wide range of uncertainty in the spatial distribution of sea ice when compared against satellite measurements and reanalysis data. Our metrics expose common and individual regional model biases, which sea ice temporal analyses alone do not capture. We identify large ice edge and ice thickness errors in Arctic sub-regions, implying possible model specific limitations in or lack of representation of some key physical processes. We postulate that many of them could be related to the oceanic forcing, especially in the marginal and shelf seas, where seasonal sea ice changes are not adequately simulated. We therefore conclude that an individual model’s ability to represent the observed/reanalysis spatial distribution still remains a challenge. We propose the spatial analysis metrics as useful tools to diagnose model limitations, narrow down possible processes affecting them, and guide future model improvements critical to the representation and projections of Arctic climate change.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRSDate not supplied

Evaluating Lunar Descent and Landing Performance From a Near Rectilinear Halo Orbit Using Linear Covariance Resetting Techniques

Upcoming lunar programs are striving the achieve precision landing in a safe and robust manner. Various elements impact this mission objective ranging from on-orbit operations with ground station tracking to incorporating relative sensors with hazard detection and avoidance (HDA) to support the final approach and landing phase. Modeling the impacts of ground tracking, trajectory replanning, relative navigation sensors, and particularly a potential HDA system on the integrated closed-loop GN\&C system performance poses a unique challenge due to the complexity and interaction with multiple facets of the vehicle including the trajectory design, sensing hardware, navigation system, guidance and targeting, and the overall mission concept of operations. This paper outlines techniques to systematically analyze and compare the performance impacts of ground tracking and replanning and an HDA system where the onboard navigation errors are reset or uploaded from an external source and the vehicle's reference trajectory is regenerated requiring the system dispersions to also be reset to reflect this in-flight profile adjustment. To illustrate the application of these general techniques for analyzing the performance impacts due to incorporating these resetting events, they are demonstrated with a human lunar descent and landing scenario starting from a near rectilinear halo orbit (NRHO) until the vehicle precisely reaches its predetermined landing site on the lunar surface. Performance metrics such as inertial and relative navigation errors, trajectory dispersions, footprint dispersions, and propellant usage are provided.

GN&C↗
NASA NTRSDate not supplied

Space Needle Returns

STEP is imported into Engineering Sketch Pad. Some bodies where slightly scaled and translated in OpenCSM to create a manifold solid for the downstream meshing process. The braces at the base and columns around the core are omitted because their solids are malformed or created nonmanifold intersections. EGADS provides an initial tessellation of the surface. refine adapted the surface mesh to a curvature and feature size metric. TetGen initially filled the volume. The TetGen mesh is adapted to the Spalding Law of the Wall u+ with refine to provide the initial mesh for flow solution. Solution-based mesh adaptation is performed where FUN3D-FV computes the flow solution with the Reynolds-averaged Navier-Stokes equations coupled to the Spalart-Allmaras turbulence model. The freestream Mach number is 4 approaching 40° from the central axis of the Space Needle. The volume and surface mesh is adapted with refine to reduce estimated interpolation error in Mach number via the multiscale metric. The adapted mesh implicitly resolves the boundary layers, shocks, and expansions. The surface mesh is shown for the lee side with a slice through the volume on the left. Computational schlieren in the lower right shows density variations. A slice of the mesh is colored with Mach number in the upper right where mesh with freestream Mach number is not rendered. The volume mesh contains 64 million vertices. A NASA worm logo is sketched and extruded into a solid in OpenCSM. The worm is unioned to the Space Needle roof to produce the inset mesh image.

mesh↗
NASA NTRS2004

Robust Timing Synchronization in Aeronautical Mobile Communication Systems

This work details a study of robust synchronization schemes suitable for satellite to mobile aeronautical applications. A new scheme, the Modified Sliding Window Synchronizer (MSWS), is devised and compared with existing schemes, including the traditional Early-Late Gate Synchronizer (ELGS), the Gardner Zero-Crossing Detector (GZCD), and the Sliding Window Synchronizer (SWS). Performance of the synchronization schemes is evaluated by a set of metrics that indicate performance in digital communications systems. The metrics are convergence time, mean square phase error (or root mean-square phase error), lowest SNR for locking, initial frequency offset performance, midstream frequency offset performance, and system complexity. The performance of the synchronizers is evaluated by means of Matlab simulation models. A simulation platform is devised to model the satellite to mobile aeronautical channel, consisting of a Quadrature Phase Shift Keying modulator, an additive white Gaussian noise channel, and a demodulator front end. Simulation results show that the MSWS provides the most robust performance at the cost of system complexity. The GZCD provides a good tradeoff between robustness and system complexity for communication systems that require high symbol rates or low overall system costs. The ELGS has a high system complexity despite its average performance. Overall, the SWS, originally designed for multi-carrier systems, performs very poorly in single-carrier communications systems. Table 5.1 in Section 5 provides a ranking of each of the synchronization schemes in terms of the metrics set forth in Section 4.1. Details of comparison are given in Section 5. Based on the results presented in Table 5, it is safe to say that the most robust synchronization scheme examined in this work is the high-sample-rate Modified Sliding Window Synchronizer. A close second is its low-sample-rate cousin. The tradeoff between complexity and lowest mean-square phase error determines the rankings of the Gardner Zero-Crossing Detector and both versions of the Early-Late Gate Synchronizer. The least robust models are the high and low-sample-rate Sliding Window Synchronizers. Consequently, the recommended replacement synchronizer for NASA's Advanced Air Transportation Technologies mobile aeronautical communications system is the high-sample-rate Modified Sliding Window Synchronizer. By incorporating this synchronizer into their system, NASA can be assured that their system will be operational in extremely adverse conditions. The quick convergence time of the MSWS should allow the use of high-level protocols. However, if NASA feels that reduced system complexity is the most important aspect of their replacement synchronizer, the Gardner Zero-Crossing Detector would be the best choice.

Xiong, Fu-Qin↗
DOE OSTI2021

Updates to the Regional Seismic Travel Time (RSTT) Model: 2. Path-dependent Travel-time Uncertainty

Abstract The regional seismic travel time (RSTT) model and software were developed to improve travel-time prediction accuracy by accounting for three-dimensional crust and upper mantle structure. Travel-time uncertainty estimates are used in the process of associating seismic phases to events and to accurately calculate location uncertainty bounds (i.e. event location error ellipses). We improve on the current distance-dependent uncertainty parameterization for RSTT using a random effects model to estimate slowness (inverse velocity) uncertainty as a mean squared error for each model parameter. The random effects model separates the error between observed slowness and model predicted slowness into bias and random components. The path-specific travel-time uncertainty is calculated by integrating these mean squared errors along a seismic-phase ray path. We demonstrate that event location error ellipses computed for a 90% coverage ellipse metric (used by the Comprehensive Nuclear-Test-Ban Treaty Organization International Data Centre (IDC)), and using the path-specific travel-time uncertainty approach, are more representative (median 82.5% ellipse percentage) of true location error than error ellipses computed using distance-dependent travel-time uncertainties (median 70.1%). We also demonstrate measurable improvement in location uncertainties using the RSTT method compared to the current station correction approach used at the IDC (median 74.3% coverage ellipse).

58 GEOSCIENCES↗
NASA NTRS2018

Optical Performance Prediction of the Thirty Meter Telescope after Initial Alignment Using Optical Modeling

We present an estimate of the optical performance of the Thirty Meter Telescope (TMT) after execution of the full telescope alignment plan. The TMT alignment is performed by the Global Metrology System (GMS) and the Alignment and Phasing System (APS). The GMS first measures the locations of the telescope optics and instruments as a function of elevation angle. These initial measurements will be used to adjust the optics positions and build initial elevation look-up tables. Then the telescope is aligned using starlight as the input for the APS at multiple elevation angles. APS measurements are used to refine the telescope alignment to build elevation and temperature dependent look-up tables. Due to the number of degrees of freedom in the telescope (over 10,000), the ability of the primary mirror to correct aberrations on other optics, the tight optical performance requirements and the multiple instrument locations, it is challenging to develop, test and validate these alignment procedures. In this paper, we consider several GMS and APS operational scenarios. We apply the alignment procedures to the model-generated TMT, which consists of various quasi-static errors such as polishing errors, passive supports errors, thermal and gravity deformations and installation position errors. Using an integrated optical model and Monte-Carlo framework, we evaluate the TMT’s aligned states using optical performance metrics at multiple instrument and field of view locations. The optical performance metrics include the Normalized Point Source Sensitivity (PSSN), RMS wavefront error before and after Adaptive Optics (AO) correction, pupil position change, and plate scale distortion.

Rogers, John↗
DOE OSTI2021

Evaluation of CMIP6 models in simulating the statistics of extreme precipitation over Eastern Africa

We report the Eastern Africa region experiences frequent extreme precipitation events that can cause destruction of property and environment, and loss of lives. Thus, there is a need to understand how these events may change in the future and how well the global climate models that are used to make projections can simulate precipitation extremes in this region before they can be used in downscaling or flood and drought impact assessment studies. In this work, we evaluated the ability of sixteen Coupled Model Intercomparison Project Phase 6 (CMIP6) models to simulate present-day precipitation extremes over the Eastern Africa region during the two rainy seasons (March–May and September–November). We used nine extreme precipitation indices (including seven (one) indices of wet (dry) extremes) defined by the Expert Team on Climate Change Detection and Indices. The CMIP6 models were evaluated against two gridded observation datasets: Global Precipitation Climatology Project One-Degree Daily Dataset and Tropical Rainfall Measuring Mission Multi-satellite Precipitation Analysis 3B42. Three model performance metrics (percentage bias, normalized root-mean-square error, and pattern correlation coefficient) were employed to further assess the strengths and weakness of the models. Our results show that the multi-model ensemble mean generally provides a better representation of observed precipitation and related extremes compared to individual models when considering all metrics and seasons. Several consistent biases are evident across CMIP6 models, which tend to overestimate the total-wet day precipitation and consecutive wet days, and underestimate very wet days and maximum 5-day precipitation in both seasons. Furthermore, no single model consistently performs best, model performance varies with the season and index under consideration and is generally independent of horizontal resolution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2020

Recent Developments in Tolerancing Methods for Imaging Spectrometers

We discuss detailed tolerancing methods developed for imaging spectrometers at NASA Jet Propulsion Laboratory, California Institute of Technology using the Earth Surface Mineral Dust Source Investigation (EMIT) imaging spectrometer as an illustrative example. We tolerance five metrics simultaneously: along-track response function, cross-track response function, spectral response function, spectral centroid uniformity, and spatial centroid uniformity. A method to calculate tolerancing sensitivities for each metric directly, a method to statistically combine Monte Carlo files from multiple tolerancing runs, and an example summary error budgets that communicate the key and driving tolerances for each metric are discussed. These methods facilitate rapid and semi-automated assessment of the predicted performance of imaging spectrometer systems from design through to assembly and launch life cycle, using metrics that are directly relevant to the extraction of accurate spectroscopic data from these instruments.

Mouroulis, Pantazis↗
NASA NTRS2018

Performance Metrics for the Assessment of Satellite Data Products: An Ocean Color Case Study

Performance assessment of ocean color satellite data has generally relied on statistical metrics chosen for their common usage and the rationale for selecting certain metrics is infrequently explained. Commonly reported statistics based on mean squared errors, such as the coefficient of determination (r2), root mean square error, and regression slopes, are most appropriate for Gaussian distributions without outliers and, therefore, are often not ideal for ocean color algorithm performance assessment, which is often limited by sample availability. In contrast, metrics based on simple deviations, such as bias and mean absolute error, as well as pair-wise comparisons, often provide more robust and straightforward quantities for evaluating ocean color algorithms with non-Gaussian distributions and outliers. This study uses a SeaWiFS chlorophyll-a validation data set to demonstrate a framework for satellite data product assessment and recommends a multimetric and user-dependent approach that can be applied within science, modeling, and resource management communities.

remote sensing↗
NASA NTRS2021

The Contributions of Gauge-Based Precipitation and SMAP Brightness Temperature Observations to the Skill of the SMAP Level-4 Soil Moisture Product

Soil Moisture Active Passive (SMAP) mission L-band brightness temperature (Tb) observations are routinely assimilated into the Catchment land surface model to generate Level-4 Soil Moisture (L4_SM) estimates of global surface and root-zone soil moisture at 9-km, 3-hourly resolution with ~2.5-day latency. The Catchment model in the L4_SM algorithm is driven with ¼-degree, hourly surface meteorological forcing data from the Goddard Earth Observing System (GEOS). Outside of Africa and the high latitudes, GEOS precipitation is corrected using Climate Prediction Center Unified (CPCU) gauge-based, ½-degree, daily precipitation. L4_SM soil moisture was previously shown to improve over land model-only estimates that use CPCU precipitation but no Tb assimilation (CPCU_SIM). Here, we additionally examine the skill of model-only (CTRL) and Tb assimilation-only (SMAP_DA) estimates derived without CPCU precipitation. Soil moisture is assessed versus in situ measurements in well-instrumented regions and globally through the Instrumental Variable (IV) method using independent soil moisture retrievals from the Advanced Scatterometer. At the in situ locations, SMAP_DA and CPCU_SIM have comparable soil moisture skill improvements relative to CTRL for the unbiased root-mean-square error (surface and root-zone) and correlation metrics (root-zone only). In the global average, SMAP Tb assimilation increases the surface soil moisture anomaly correlation by 0.10-0.11 compared to an increase of 0.02-0.03 from the CPCU-based precipitation corrections. The contrast is particularly strong in central Australia, where CPCU is known to have errors and observation-minus-forecast Tb residuals are larger when CPCU precipitation is used. Validation versus streamflow measurements in the contiguous U.S. reveals that CPCU precipitation provides most of the skill gained in L4_SM runoff estimates over CTRL.

SMAP↗
NASA NTRS2012

Improving operations: Metrics to Results

As a result of the mission failure of the Mars Climate Orbiter (MCO) spacecraft in 1999, the Jet Propulsion Laboratory (JPL) initiated the development of a Mission Operations Assurance (MOA) program to be implemented across all flight projects managed by JPL. One of the initiatives undertaken in 2001 was the collection of data on command file errors occurring in the operational phase of the mission. This paper defines command file errors and how and where they occur in the operations process. It also describes the problem reporting system (PRS) in use for mission operations at JPL. We examine the recent modifications to the PRS that enable the collection of metrics, specifically on command file errors. This paper discusses what the data show us since metrics have been collected for the operational missions conducted by JPL. We examine the evolution of an operational working group initiative to evaluate proximate, contributing, and root causes for the errors. As part of this discussion we see what the metrics have indicated over a decade. At the macro level, we can say that the aggregate command file error rate has been cut to roughly one third of the initial 2001 level by the end of 2011. Additionally, we explore efficient and innovative means to continually integrate the findings and recommendations from the working group back into the flight operations environment.

command file errors↗
DOE OSTI2022

Validation Analysis of Medium-Scale Methanol Pool Fire Simulated in SIERRA/Fuego [Slides]

Analysis of methanol pool fire conducted as part of validation study for SIERRA/Fuego. Results showed trends & errors consistent with related studies. Area validation metric provides way to quantify model form uncertainty. AVM shows that more work could be done to understand how model form uncertainty varies with mesh resolution. There is a possible atypical use of MAVM on time-series data. AVM shows mismatch between predicted flame height and experimental value less sensitive to variations in mixture fraction than temperature. Mismatch about experimental value also more symmetric for mixture fraction. Our analysis showed that mixture fraction is preferable for this application.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2005

Passive isolator design for jitter reduction in the Terrestrial Planet Finder Coronagraph

Terrestrial Planet Finder (TPF) is a mission to locate and study extrasolar Earth-like planets. The TPF Coronagraph (TPF-C), planned for launch in the latter half of the next decade, will use a coronagraphic mask and other optics to suppress the light of the nearby star in order to collect visible light from such planets. The required contrast ratio of 5e-11 can only be achieved by maintaining pointing accuracy to 4 milli-arcseconds, and limiting optics jitter to below 5 nm. Numerous mechanical disturbances act to induce jitter. This paper concentrates on passive isolation techniques to minimize the optical degradation introduced by disturbance sources. A passive isolation system, using compliant mounts placed at an energy bottleneck to reduce energy transmission above a certain frequency, is a low risk, flight proven design approach. However, the attenuation is limited, compared to an active system, so the feasibility of the design must be demonstrated by analysis. The paper presents the jitter analysis for the baseline TPF design, using a passive isolation system. The analysis model representing the dynamics of the spacecraft and telescope is described, with emphasis on passive isolator modeling. Pointing and deformation metrics, consistent with the TPF-C error budget, are derived. Jitter prediction methodology and results are presented. Then an analysis of the critical design parameters that drive the TPF-C jitter response is performed.

Blaurock, Carl↗
NASA NTRS2010

Configurable Multi-Purpose Processor

Advancements in technology have allowed the miniaturization of systems used in aerospace vehicles. This technology is driven by the need for next-generation systems that provide reliable, responsive, and cost-effective range operations while providing increased capabilities such as simultaneous mission support, increased launch trajectories, improved launch, and landing opportunities, etc. Leveraging the newest technologies, the command and telemetry processor (CTP) concept provides for a compact, flexible, and integrated solution for flight command and telemetry systems and range systems. The CTP is a relatively small circuit board that serves as a processing platform for high dynamic, high vibration environments. The CTP can be reconfigured and reprogrammed, allowing it to be adapted for many different applications. The design is centered around a configurable field-programmable gate array (FPGA) device that contains numerous logic cells that can be used to implement traditional integrated circuits. The FPGA contains two PowerPC processors running the Vx-Works real-time operating system and are used to execute software programs specific to each application. The CTP was designed and developed specifically to provide telemetry functions; namely, the command processing, telemetry processing, and GPS metric tracking of a flight vehicle. However, it can be used as a general-purpose processor board to perform numerous functions implemented in either hardware or software using the FPGA s processors and/or logic cells. Functionally, the CTP was designed for range safety applications where it would ultimately become part of a vehicle s flight termination system. Consequently, the major functions of the CTP are to perform the forward link command processing, GPS metric tracking, return link telemetry data processing, error detection and correction, data encryption/ decryption, and initiate flight termination action commands. Also, the CTP had to be designed to survive and operate in a launch environment. Additionally, the CTP was designed to interface with the WFF (Wallops Flight Facility) custom-designed transceiver board which is used in the Low Cost TDRSS Transceiver (LCT2) also developed by WFF. The LCT2 s transceiver board demodulates commands received from the ground via the forward link and sends them to the CTP, where they are processed. The CTP inputs and processes data from the inertial measurement unit (IMU) and the GPS receiver board, generates status data, and then sends the data to the transceiver board where it is modulated and sent to the ground via the return link. Overall, the CTP has combined processing with the ability to interface to a GPS receiver, an IMU, and a pulse code modulation (PCM) communication link, while providing the capability to support common interfaces including Ethernet and serial interfaces boarding a relatively small-sized, lightweight package.

Valencia, J. Emilio↗