SEARCH · Engineering Papers

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS2012

An OSSE on Mesoscale Model Assimilation of Simulated HIRAD-Observed Hurricane Surface Winds

The hazards of landfalling hurricanes are well known, but progress on improving the intensity forecasts of these deadly storms at landfall has been slow. Many cite a lack of high-resolution data sets taken inside the core of a hurricane, and the lack of reliable measurements in extreme conditions near the surface of hurricanes, as possible reasons why even the most state-of-the-art forecasting models cannot seem to forecast intensity changes better. The Hurricane Imaging Radiometer (HIRAD) is a new airborne microwave remote sensor for observing hurricanes, and is operated and researched by NASA Marshall Space Flight Center in partnership with the NOAA Atlantic Oceanographic and Meteorological Laboratory/Hurricane Research Division, the University of Central Florida, the University of Michigan, and the University of Alabama in Huntsville. This instrument?s purpose is to study the wind field of a hurricane, specifically observing surface wind speeds and rain rates, in what has traditionally been the most difficult areas for other instruments to study; the high wind and heavy rain regions. Dr. T. N. Krishnamurti has studied various data assimilation techniques for hurricane and monsoon rain rates, and this study builds off of results obtained from utilizing his style of physical initializations of rainfall observations, but obtaining reliable observations in heavy rain regions has always presented trouble to our research of high-resolution rainfall forecasting. Reliable data from these regions at such a high resolution and wide swath as HIRAD provides is potentially very valuable to mesoscale forecasting of hurricane intensity. This study shows how the data assimilation technique of Ensemble Kalman Filtering (EnKF) in the Weather Research and Forecasting (WRF) model can be used to incorporate wind, and later rain rate, data into a mesoscale model forecast of hurricane intensity. The study makes use of an Observing System Simulation Experiment (OSSE) with a simulated HIRAD dataset sampled during a hurricane and uses EnKF to forecast the track and intensity prediction of the hurricane. Comparisons to truth and error metrics are used to assess the model?s forecast performance.

Albers, Cerese↗

NASA NTRS2017

Using Dispersed Modes During Model Correlation

The model correlation process for the modal characteristics of a launch vehicle is well established. After a test, parameters within the nominal model are adjusted to reflect structural dynamics revealed during testing. However, a full model correlation process for a complex structure can take months of man-hours and many computational resources. If the analyst only has weeks, or even days, of time in which to correlate the nominal model to the experimental results, then the traditional correlation process is not suitable. This paper describes using model dispersions to assist the model correlation process and decrease the overall cost of the process. The process creates thousands of model dispersions from the nominal model prior to the test and then compares each of them to the test data. Using mode shape and frequency error metrics, one dispersion is selected as the best match to the test data. This dispersion is further improved by using a commercial model correlation software. In the three examples shown in this paper, this dispersion based model correlation process performs well when compared to models correlated using traditional techniques and saves time in the post-test analysis.

Stewart, Eric C.↗

NASA NTRS2020

An AeroCom–AeroSat study: intercomparison of satellite AOD datasets for aerosol model evaluation

To better understand and characterize current uncertainties in the important observational constraint of climate models of aerosol optical depth (AOD), we evaluate and intercompare 14 satellite products, representing nine different retrieval algorithm families using observations from five different sensors on six different platforms. The satellite products (super-observations consisting of 1°×1° daily aggregated retrievals drawn from the years 2006, 2008 and 2010) are evaluated with AErosol RObotic NETwork (AERONET) and Maritime Aerosol Network (MAN) data. Results show that different products exhibit different regionally varying biases (both under- and overestimates) that may reach ±50 %, although a typical bias would be 15 %–25 % (depending on the product). In addition to these biases, the products exhibit random errors that can be 1.6 to 3 times as large. Most products show similar performance, although there are a few exceptions with either larger biases or larger random errors. The intercomparison of satellite products extends this analysis and provides spatial context to it. In particular, we show that aggregated satellite AOD agrees much better than the spatial coverage (often driven by cloud masks) within the 1°×1° grid cells. Up to ∼50 % of the difference between satellite AOD is attributed to cloud contamination. The diversity in AOD products shows clear spatial patterns and varies from 10 % (parts of the ocean) to 100 % (central Asia and Australia). More importantly, we show that the diversity may be used as an indication of AOD uncertainty, at least for the better performing products. This provides modellers with a global map of expected AOD uncertainty in satellite products, allows assessment of products away from AERONET sites, can provide guidance for future AERONET locations and offers suggestions for product improvements. We account for statistical and sampling noise in our analyses. Sampling noise, variations due to the evaluation of different subsets of the data, causes important changes in error metrics. The consequences of this noise term for product evaluation are discussed.

Aerosol↗

NASA NTRSDate not supplied

Autogenous Pressurization of a Cryogenic Tank Using Computational Fluid Dynamics

Well-validated simulation tools can be used to predict long-term storage and transfer of cryogenic propellants, which are essential to NASA’s mission plan to return to the Moon and continue to Mars. Autogenous pressurization of propellant tanks is used to supply rocket engine turbopumps with pressurized liquid fuel and oxidizer, preventing cavitation. Additionally, autogenous pressurization can be used to pressurize propellant tanks to support on-orbit propellant transfer. In lieu of expensive tests conducted on-orbit, accurate predictive computational models of these processes can be used to reduce system and propellant mass as well as mission risk. Using simulation tools also reduces the cost of analyzing and developing this technology. This study presents a multiphase computational fluid dynamics model capable of simulating autogenous pressurization of a large cryogenic tank using the commercial code STAR-CCM+. Experimental data from a full-scale tank pressurization test under terrestrial gravity is compared to simulation results using transient error metrics. Comparisons show good agreement and give confidence in using these and other validated simulation tools to develop cryogenic pressurization systems.

Cryogenics↗

NASA NTRSDate not supplied

Autogenous Pressurization of a Cryogenic Tank using Computational Fluid Dynamics

cryogenics↗

NASA NTRSDate not supplied

An L1 Adaptive Control Augmentation for a Lift Plus-Cruise Vehicle

This paper presents anL1adaptive control augmentation for a Lift-Plus-Cruise (L+C)vehicle. This class of vehicles operates in three flight modes with different dynamic behavior: vertical, transition, and forward flight. A robust uniform controller is used as a baseline to stabilize the system throughout these flight modes. The uniform controller is a linear control law designed around trim conditions of the aircraft and includes control allocation to achieve the desired forces and moments on the vehicle. TheL1control augmentation is designed for each of these trim conditions to compensate for the nonlinear time- and state-dependent uncertainties in the vehicle dynamics. The augmented control output is then added to the desired force and moment commands on the vehicle. Simulation results demonstrate the effectiveness of control augmentation for reducing the effects of unmodeled dynamics, reduced actuator effectiveness, and time-dependent disturbances. Effectiveness is demonstrated through tracking error metrics.

Andrew Patterson↗

NASA NTRS1970

MSFN unified S-band metric tracking capabilities

Error sources affecting metric tracking capability of unified S band

Salzberg, I.↗

NASA NTRSDate not supplied

Fusing GeoNEX and VIIRS Surface BRDF Retrievals: Exploring a GEO-LEO Synergy

The Bidirectional Reflectance Distribution Function or BRDF, which describes the dependency of surface reflectance on the illumination-view geometries, are the foundation of many high-level satellite products for terrestrial and aquatic system monitoring. The latest geostationary sensors like GOES ABI provide high frequent (~10 minutes) observations of the Earth surface that feature continuously changing sun angles, allowing us to retrieve surface BRDF with dedicated atmospheric correction algorithms like MAIAC (Multi-Angle Implementation of Atmospheric Correction). For mid-latitude locations, because geostationary satellites have fixed view angles in the back-scattering directions, the angular sampling of surface BRDF by GEO sensors is not comprehensive. This study explores a GEO-LEO synergy to address this issue. We first extract concurrent GeoNEX and VIIRS BRDF data with the best quality (cloud-free and low aerosol loading) at chosen AERONET sites. We then compare the magnitude and the shape factors of the two set of BRDF parameters as well as their variations through the season. We calculate the “distances” between the GeoNEX and VIIRS BRDF by using them to cross-predict the top-of-atmosphere reflectance measured by their counterpart and evaluating the corresponding prediction errors. This metric allows us to derive a set of optimized BRDF parameters that minimize such distances or prediction errors, which are considered as the fused BRDF result. We validate the algorithm with reserved AERONET data and then apply it to generate the GEO-LEO BRDF synergy over CONUS. We expect the fused BRDF to have reduced uncertainties as compared to the source GeoNEX or VIIRS data and may find broadly application in deriving other high-level satellite products.

Geostationary satellite↗

NASA NTRS1991

Systematic methods for knowledge acquisition and expert system development

Nine cooperating rule-based systems, collectively called AUTOCREW, were designed to automate functions and decisions associated with a combat aircraft's subsystem. The organization of tasks within each system is described; performance metrics were developed to evaluate the workload of each rule base, and to assess the cooperation between the rule-bases. Each AUTOCREW subsystem is composed of several expert systems that perform specific tasks. AUTOCREW's NAVIGATOR was analyzed in detail to understand the difficulties involved in designing the system and to identify tools and methodologies that ease development. The NAVIGATOR determines optimal navigation strategies from a set of available sensors. A Navigation Sensor Management (NSM) expert system was systematically designed from Kalman filter covariance data; four ground-based, a satellite-based, and two on-board INS-aiding sensors were modeled and simulated to aid an INS. The NSM Expert was developed using the Analysis of Variance (ANOVA) and the ID3 algorithm. Navigation strategy selection is based on an RSS position error decision metric, which is computed from the covariance data. Results show that the NSM Expert predicts position error correctly between 45 and 100 percent of the time for a specified navaid configuration and aircraft trajectory. The NSM Expert adapts to new situations, and provides reasonable estimates of hybrid performance. The systematic nature of the ANOVA/ID3 method makes it broadly applicable to expert system design when experimental or simulation data is available.

Belkin, Brenda L.↗

NASA NTRS2008

The Super Tuesday Outbreak: Forecast Sensitivities to Single-Moment Microphysics Schemes

Forecast precipitation and radar characteristics are used by operational centers to guide the issuance of advisory products. As operational numerical weather prediction is performed at increasingly finer spatial resolution, convective precipitation traditionally represented by sub-grid scale parameterization schemes is now being determined explicitly through single- or multi-moment bulk water microphysics routines. Gains in forecasting skill are expected through improved simulation of clouds and their microphysical processes. High resolution model grids and advanced parameterizations are now available through steady increases in computer resources. As with any parameterization, their reliability must be measured through performance metrics, with errors noted and targeted for improvement. Furthermore, the use of these schemes within an operational framework requires an understanding of limitations and an estimate of biases so that forecasters and model development teams can be aware of potential errors. The National Severe Storms Laboratory (NSSL) Spring Experiments have produced daily, high resolution forecasts used to evaluate forecast skill among an ensemble with varied physical parameterizations and data assimilation techniques. In this research, high resolution forecasts of the 5-6 February 2008 Super Tuesday Outbreak are replicated using the NSSL configuration in order to evaluate two components of simulated convection on a large domain: sensitivities of quantitative precipitation forecasts to assumptions within a single-moment bulk water microphysics scheme, and to determine if these schemes accurately depict the reflectivity characteristics of well-simulated, organized, cold frontal convection. As radar returns are sensitive to the amount of hydrometeor mass and the distribution of mass among variably sized targets, radar comparisons may guide potential improvements to a single-moment scheme. In addition, object-based verification metrics are evaluated for their utility in gauging model performance and QPF variability.

Molthan, Andrew L.↗

NASA NTRS2012

Interuser Interference Analysis for Direct-Sequence Spread-Spectrum Systems Part I: Partial-Period Cross-Correlation

This presentation discusses an analysis approach to evaluate the interuser interference for Direct-Sequence Spread-Spectrum (DSSS) Systems for Space Network (SN) Users. Part I of this analysis shows that the correlation property of pseudo noise (PN) sequences is the critical factor which determines the interuser interference performance of the DSSS system. For non-standard DSSS systems in which PN sequence s period is much larger than one data symbol duration, it is the partial-period cross-correlation that determines the system performance. This study reveals through an example that a well-designed PN sequence set (e.g. Gold Sequence, in which the cross-correlation for a whole-period is well controlled) may have non-controlled partial-period cross-correlation which could cause severe interuser interference for a DSSS system. Since the analytical derivation of performance metric (bit error rate or signal-to-noise ratio) based on partial-period cross-correlation is prohibitive, the performance degradation due to partial-period cross-correlation will be evaluated using simulation in Part II of this analysis in the future.

Ni, Jianjun (David)↗

NASA NTRS2015

Comparing Anisotropic Output-Based Grid Adaptation Methods by Decomposition

Anisotropic grid adaptation is examined by decomposing the steps of flow solution, ad- joint solution, error estimation, metric construction, and simplex grid adaptation. Multiple implementations of each of these steps are evaluated by comparison to each other and expected analytic results when available. For example, grids are adapted to analytic metric fields and grid measures are computed to illustrate the properties of multiple independent implementations of grid adaptation mechanics. Different implementations of each step in the adaptation process can be evaluated in a system where the other components of the adaptive cycle are fixed. Detailed examination of these properties allows comparison of different methods to identify the current state of the art and where further development should be targeted.

Park, Michael A.↗

NASA NTRS2018

Unstructured Grid Adaptation and Solver Technology for Turbulent Flows

Unstructured grid adaptation is a tool to control Computational Fluid Dynamics (CFD) discretization error. However, adaptive grid techniques have made limited impact on production analysis workflows where the control of discretization error is critical to obtaining reliable simulation results. Issues that prevent the use of adaptive grid methods are identified by applying unstructured grid adaptation methods to a series of benchmark cases. Once identified, these challenges to existing adaptive workflows can be addressed. Unstructured grid adaptation is evaluated for test cases described on the Turbulence Modeling Resource (TMR) web site, which documents uniform grid refinement of multiple schemes. The cases are turbulent flow over a Hemisphere Cylinder and an ONERA M6Wing. Adaptive grid force and moment trajectories are shown for three integrated grid adaptation processes with Mach interpolation control and output error based metrics. The integrated grid adaptation process with a finite element (FE) discretization produced results consistent with uniform grid refinement of fixed grids. The integrated grid adaptation processes with finite volume schemes were slower to converge to the reference solution than the FE method. Metric conformity is documented on grid/metric snapshots for five grid adaptation mechanics implementations. These tools produce anisotropic boundary conforming grids requested by the adaptation process.

Park, Michael A.↗

NASA NTRSDate not supplied

A Novel Machine Learning Method for Surface PM2.5 Estimations from Geostationary Satellites

Particulate matter (PM) with a diameter of less or equal to 2.5 μm, known as PM , affects human health as it penetrates the respiratory system. The Environmental Protection Agency (EPA) measures the atmospheric concentration of PM using air quality monitors stationed throughout the Continental United States (CONUS). Such measurements are points on a spatial domain and therefore, might not be representative of the air quality at nearby areas considering that the composition of the atmosphere is highly variable from place to place. Satellite based AOD permits a spatially uniform means of estimating PM and new geostationary satellites provide high temporal and spatial resolution estimation of AOD. However, the concentration of PM is non-linearly dependent on other atmospheric parameters that include relative humidity, temperature, and height of the planetary boundary layer. This information may be estimated at similar spatial and temporal resolutions as AOD from numerical modeling such as from the National Oceanic and Atmospheric Administration’s (NOAA) High Resolution Rapid Refresh (HRRR) model which resolves near real-time atmospheric conditions over the CONUS. The estimation of PM concentration is a multi-parametric problem that considers the effect of temporal dependencies among the different parameters. Deep learning approaches are appropriate for such complex estimation problems as they intrinsically capture relations among multiple non-linear parameters. This study compares deep-learning methods to traditional regression analysis to demonstrate the capabilities of these methods in predicting PM2.5 concentrations. Additionally, a novel ensemble learning approach is employed to identify scientific processes that could further improve the estimation of PM concentration. Utilizing Long Short-Term Memory (LSTM) neural networks, which are suitable for multivariate time series estimation problems as they are capable of learning long-term dependencies, individual models are created for each EPA station and trained on the aforementioned dataset collocated over each station. Individual station models are merged if the model's performance is improved by reducing the root mean squared error (RMSE) metric. This ensemble training method ultimately reduces the RMSE value. Evaluation of these results provide insights into physical processes and related observable parameters that may contribute to PM concentrations. Identified parameters evaluated to be statistically different between the merged and unmerged models are expected to improve overall performance. These new parameters are then utilized for reevaluation of the deep learning methods with an extreme gradient boosting model with an RMSE of 5.5 providing the best results.

George Priftis↗

NASA NTRS1983

Software development predictors, error analysis, reliability models and software metric analysis

The use of dynamic characteristics as predictors for software development was studied. It was found that there are some significant factors that could be useful as predictors. From a study on software errors and complexity, it was shown that meaningful results can be obtained which allow insight into software traits and the environment in which it is developed. Reliability models were studied. The research included the field of program testing because the validity of some reliability models depends on the answers to some unanswered questions about testing. In studying software metrics, data collected from seven software engineering laboratory (FORTRAN) projects were examined and three effort reporting accuracy checks were applied to demonstrate the need to validate a data base. Results are discussed.

Basili, Victor↗

NASA NTRS2023

Improving the MODIS LAI Compositing Using Prior Time-Series Information

The Moderate Resolution Imaging Spectroradiometer (MODIS) long-term leaf area index (LAI) products have significantly contributed to global energy fluxes, climate change, and biogeochemistry research. However, the maximum fraction of photosynthetically active radiation absorbed by vegetation (Max-FPAR) compositing strategy of the Collection 6 (C6) products dictates that the main or backup algorithm is always triggered by observations of different quality, which indirectly causes the observed instability in the LAI time-series. Based on MODIS daily LAI retrievals, this study develops a prior knowledge time-series compositing algorithm (PKA) using a linear kernel driven (LKD) model. Our results show that the newly proposed PKA can significantly improve the LAI composites compared to the Max-FPAR strategy using ground-based observations for validation. We found that the PKA performs better than Max-FPAR in various aspects (different sites, seasons, and retrieval index (RI) ranges), with R2 increasing from 0.69 to 0.76 and root means square error (RMSE) decreasing from 1.01 to 0.84 compared to GBOV ground truth. The same improvement was shown for the ground truth LAIs measured at the Honghe and Hailun sites in northeastern China, with R2 increasing from 0.23 to 0.41 and RMSE decreasing from 1.27 to 1.25. In addition, three newly proposed temporal uncertainty metrics (time-series stability, TSS and time-series anomaly, TSA and reconstruction error metric, RE (the proximity to the main RT-based retrievals)) were applied to compare the stability of LAI time-series before and after PKA implementation. We found that the time series stability of PKA LAI was improved, the time series anomalies were reduced, and the retrieval rates of the main algorithm were also greatly enhanced compared to Max-FPAR LAI. A case intercomparison for Max-FPAR-MODIS, Max-FPAR-VIIRS (Visible Infrared Imager Radiometer Suite), and PKA-MODIS LAIs in the Amazon Forest region showed that the PKA is also effective in improving the LAI retrieval over large regions with few qualified observations due to poor atmospheric conditions (RE decreased from 2.37/2.35 (Max-FPAR-MODIS/Max-FPAR-VIIRS) to 2.25 (PKA-MODIS) and RI increased from 61.94%/59.62% to 66.88%). The same improvement was seen in the BELMANIP 2.1 sites for almost all biomes except deciduous broadleaf forest, where the RE decreased from 1.85/2.13 to 1.15 overall. We note that the PKA has the potential to be easily implemented in the operational algorithms of subsequent MODIS and MODIS-like LAI Collections.

MODIS↗

NASA NTRS1992

Deployable reflector antenna performance optimization using automated surface correction and array-feed compensation

Methods for increasing the electromagnetic (EM) performance of reflectors with rough surfaces were tested and evaluated. First, one quadrant of the 15-meter hoop-column antenna was retrofitted with computer-driven and controlled motors to allow automated adjustment of the reflector surface. The surface errors, measured with metric photogrammetry, were used in a previously verified computer code to calculate control motor adjustments. With this system, a rough antenna surface (rms of approximately 0.180 inch) was corrected in two iterations to approximately the structural surface smoothness limit of 0.060 inch rms. The antenna pattern and gain improved significantly as a result of these surface adjustments. The EM performance was evaluated with a computer program for distorted reflector antennas which had been previously verified with experimental data. Next, the effects of the surface distortions were compensated for in computer simulations by superimposing excitation from an array feed to maximize antenna performance relative to an undistorted reflector. Results showed that a 61-element array could produce EM performance improvements equal to surface adjustments. When both mechanical surface adjustment and feed compensation techniques were applied, the equivalent operating frequency increased from approximately 6 to 18 GHz.

Schroeder, Lyle C.↗

NASA NTRS2018

LUVOIR Primary Mirror Segment Alignment Control With Joint Laser Metrology and Segment Edge Sensing

An approach is developed for the alignment and stability maintenance of the LUVOIR segmented primary mirror using a segment state estimation and wavefront control method based on a hybrid segment motion sensing architecture of laser truss metrology and segment edge sensors. Our current computer model was generated for LUVOIR Architecture Option A with a 15-meter aperture, 120segment primary mirror. The methodology and simulation results will be presented and analyzed. JPL has a long history of technology development in laser metrology and edge sensors, including work in SIM [7], Keck and TMT [8], CCAT [3] and LUVOIR [1]. We will discuss our current efforts of LUVOIR laser metrology and edge-sensor models development, showing sensitivities of sensor measurements to various mirror eigenmodes, removing global modes and strengthening weak modes by performing joint (hybrid) laser-metrology and edge sensing. We will define and derive an important performance metric called wavefront error multiplier (WEM), and show that WEM provides a simple link between sensor errors and the closed-loop (controlled) system wavefront error. We will show WEM values for several hybrid sensor configuration options studied. We will discuss an algorithm for mirror shape control and maintenance through segment state and wavefront estimations using joint edge-metrology sensing. We will compare simulated performance of mirror state estimation, wavefront estimation and wavefront control based on joint edge-metrology sensing among several sensor configurations, and show the impact of sensor error distributions on the segmented mirror alignment performance. Mirror shape control performance will be also be evaluated in terms of imaging contrast between inner working angles (IWA) and outer working angles (OWA) of a LUVOIR coronagraph. The research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. LUVOIR is a mission concept under study for the next astronomy Decadal Survey—the information presented in this paper is pre-decisional and is provided for planning and discussion purposes only.

Shelton, Chris↗