Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
NASA NTRS2023

Atmospheric Correction Inter-comparison eXercise, ACIX-II Land: An Assessment of Amospheric Correction Processors for Landsat 8 and Sentinel-2 Over Land

The correction of the atmospheric effects on optical satellite images is essential for quantitative and multi-temporal remote sensing applications. In order to study the performance of the state-of-the-art methods in an integrated way, a voluntary and open-access benchmark Atmospheric Correction Inter-comparison eXercise (ACIX) was initiated in 2016 in the frame of Committee on Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). The first exercise was extended in a second edition wherein twelve atmospheric correction (AC) processors, a substantially larger testing dataset and additional validation metrics were involved. The sites for the inter-comparison analysis were defined by investigating the full catalogue of the Aerosol Robotic Network (AERONET) sites for coincident measurements with satellites' overpass. Although there were more than one hundred sites for Copernicus Sentinel-2 and Landsat 8 acquisitions, the analysis presented in this paper concerns only the common matchups amongst all processors, reducing the number to 79 and 62 sites respectively. Aerosol Optical Depth (AOD) and Water Vapour (WV) retrievals were consequently validated based on the available AERONET observations. The processors mostly succeeded in retrieving AOD for relatively light to medium aerosol loading (AOD < 0.2) with uncertainties <0.08, while the overall uncertainty values were typically 0.23 ± 0.15. Better performances were observed for WV retrievals with >90% of the results falling within the suggested empirical specifications and with the Root Mean Square Error (RMSE) being mostly <0.25 g/cm2. Regarding Surface Reflectance (SR) validation two main approaches were followed. For the first one, a simulated SR reference dataset was computed over all of the test sites by using the 6SV (Second Simulation of the Satellite Signal in the Solar Spectrum vector code) full radiative transfer modelling (RTM) and AERONET measurements for the required aerosol variables and water vapour content. The performance assessment demonstrated that the retrievals were not biased for most of the bands. The uncertainties ranged from approximately 0.003 to 0.01 (excluding B01) for the best performing processors in both sensors' analyses. For the second one, measurements from the radiometric calibration network RadCalNet over La Crau (France) and Gobabeb (Namibia) were involved in the validation. The performance of the processors was in general consistent across all bands for both sensors and with low standard deviations (<0.04) between on-site and estimated surface reflectance. Overall, our study provides a good insight of AC algorithms' performance to developers and users, pointing out similarities and differences for AOD, WV and SR retrievals. Such validation though still lacks of ground-based measurements of known uncertainty to better assess and characterize the uncertainties in SR retrievals.

Atmospheric correction↗
NASA NTRS2017

Mind the Gap: Exploring the Underground of the NASA Space Cancer Risk Model

The REID quantifies the lifetime risk of death from radiation-induced cancer in an exposed astronaut. The NASA Space Cancer Risk (NSCR) 2012 mode incorporates elements from physics, biology, epidemiology, and statistics to generate the REID distribution. The current model quantifies the space radiation environment, radiation quality, and dose-rate effects to estimate a NASA-weighted dose. This weighted dose is mapped to the excess risk of radiation-induced cancer mortality from acute exposures to gamma rays and then transferred to an astronaut population. Finally, the REID is determined by integrating this risk over the individual's lifetime. The calculated upper 95% confidence limit of the REID is used to restrict an astronaut's permissible mission duration (PMD) for a proposed mission. As a statistical quantity characterized by broad, subjective uncertainties, REID estimates for space missions result in wide distributions. Currently, the upper 95% confidence level is over 350% larger than the mean REID value, which can severely limit an astronaut's PMD. The model incorporates inputs from multiple scientific disciplines in the risk estimation process. Physics and particle transport models calculate how radiation moves through space, penetrates spacecraft, and makes its way to the human beings onboard. Epidemiological studies of exposures from atomic bombings, medical treatments, and power plants are used to quantify health risks from acute and chronic low linear energy transfer (LET) ionizing radiation. Biological studies in cellular and animal models using radiation at various LETs and energies inform quality metrics for ions present in space radiation. Statistical methodologies unite these elements, controlling for mathematical and scientific uncertainty and variability. Despite current progress, these research platforms contain knowledge gaps contributing to the large uncertainties still present in the model. The NASA Space Radiation Program Element (SRPE) defines the knowledge gaps that impact our understanding of the cancer risks. These gaps are outlined in NASA's Human Research Roadmap [4], which identifies the research questions and actions recommended for reducing the uncertainty in the current NSCR model and for formulation of future models. The greatest contributors to uncertainty in the current model include radiation quality, dose rate effects, and the transfer of exposure-based risk from other populations to an astronaut population. Future formulations of the risk model may benefit from including other potential sources of uncertainty such as space dosimetry, errors in human epidemiology data, and the impact of microgravity and other spaceflight stressors. Here, we discuss the current capabilities of the NSCR-2012 model and several immediate research needs, highlighting areas expected to have an operational impact on the current model schema. The following subway-style route map outlines the NSCR-2012 model (Green Line), emphasizing the research gaps in the Human Research Roadmap for risk of radiation-induced carcinogenesis (Stops on Dashed Lines). The map diagrams how these research gaps feed specific portions of the model.

Chappell, L. J.↗
NASA NTRS2019

Application of Machine Learning Techniques to Delay Tolerant Network Routing

This dissertation discusses several machine learning techniques to improve routing in delay tolerant networks (DTNs). These are networks in which there may be long one-way trip times, asymmetric links, high error rates, and deterministic as well as non-deterministic loss of contact between network nodes, such as interplanetary satellite networks, mobile ad hoc networks and wireless sensor networks. This work uses historical network statistics to train a multi-label classifier to predict reliable paths through the network. In addition, a clustering technique is used to predict future mobile node locations. Both of these techniques are used to reduce the consumption of resources such as network bandwidth, memory and data storage that is required by replication routing methods often used in opportunistic DTN environments. Thesis contributions include: an emulation tool chain developed to create a DTN test bed for machine learning, the network and software architecture for a machine learning based routing method, the development and implementation of classification and clustering techniques and performance evaluation in terms of machine learning and routing metrics.

Dudukovich, Rachel M.↗
NASA NTRS2017

Similarity Assessment of Land Surface Model Outputs in the North American Land Data Assimilation System

Multimodel ensembles are often used to produce ensemble mean estimates that tend to have increased simulation skill over any individual model output. If multimodel outputs are too similar, an individual LSM would add little additional information to the multimodel ensemble, whereas if the models are too dissimilar, it may be indicative of systematic errors in their formulations or configurations. The article presents a formal similarity assessment of the North American Land Data Assimilation System (NLDAS) multimodel ensemble outputs to assess their utility to the ensemble, using a confirmatory factor analysis. Outputs from four NLDAS Phase 2 models currently running in operations at NOAA/NCEP and four new/ upgraded models that are under consideration for the next phase of NLDAS are employed in this study. The results show that the runoff estimates from the LSMs were most dissimilar whereas the models showed greater similarity for root zone soil moisture, snow water equivalent, and terrestrial water storage. Generally, the NLDAS operational models showed weaker association with the common factor of the ensemble and the newer versions of the LSMs showed stronger association with the common factor, with the model similarity increasing at longer time scales. Trade-offs between the similarity metrics and accuracy measures indicated that the NLDAS operational models demonstrate a larger span in the similarity-accuracy space compared to the new LSMs. The results of the article indicate that simultaneous consideration of model similarity and accuracy at the relevant time scales is necessary in the development of multimodel ensemble.

land surface models↗
NASA NTRS2011

Finding Every Root of a Broad Class of Real, Continuous Functions in a Given Interval

One of the most pervasive needs within the Deep Space Network (DSN) Metric Prediction Generator (MPG) view period event generation is that of finding solutions to given occurrence conditions. While the general form of an equation expresses equivalence between its left-hand and right-hand expressions, the traditional treatment of the subject subtracts the two sides, leaving an expression of the form Integral of(x) = 0. Values of the independent variable x satisfying this condition are roots, or solutions. Generally speaking, there may be no solutions, a unique solution, multiple solutions, or a continuum of solutions to a given equation. In particular, all view period events are modeled as zero crossings of various metrics; for example, the time at which the elevation of a spacecraft reaches its maximum value, as viewed from a Deep Space Station (DSS), is found by locating that point at which the derivative of the elevation function becomes zero. Moreover, each event type may have several occurrences within a given time interval of interest. For example, a spacecraft in a low Moon orbit will experience several possible occultations per day, each of which must be located in time. The MPG is charged with finding all specified event occurrences that take place within a given time interval (or pass ), without any special clues from operators as to when they may occur, for the entire spectrum of missions undertaken by the DSN. For each event type, the event metric function is a known form that can be computed for any instant within the interval. A method has been created for a mathematical root finder to be capable of finding all roots of an arbitrary continuous function, within a given interval, to be subject to very lenient, parameterized assumptions. One assumption is that adjacent roots are separated at least by a given amount, xGuard. Any point whose function value is less than ef in magnitude is considered to be a root, and the function values at distances xGuard away from a root are larger than ef, unless there is another root located in this vicinity. A root is considered found if, during iteration, two root candidates differ by less than a pre-specified ex, and the optimum cubic polynomial matching the function at the end and at two interval points (that is within a relative error fraction L at its midpoint) is reliable in indicating whether the function has extrema within the interval. The robustness of this method depends solely on choosing these four parameters that control the search. The roots of discontinuous functions were also found, but at degraded performance.

Tausworthe, Robert C.↗
NASA NTRS2012

Reynolds-Averaged Turbulence Model Assessment for a Highly Back-Pressured Isolator Flowfield

The use of computational fluid dynamics in scramjet engine component development is widespread in the existing literature. Unfortunately, the quantification of model-form uncertainties is rarely addressed with anything other than sensitivity studies, requiring that the computational results be intimately tied to and calibrated against existing test data. This practice must be replaced with a formal uncertainty quantification process for computational fluid dynamics to play an expanded role in the system design, development, and flight certification process. Due to ground test facility limitations, this expanded role is believed to be a requirement by some in the test and evaluation community if scramjet engines are to be given serious consideration as a viable propulsion device. An effort has been initiated at the NASA Langley Research Center to validate several turbulence closure models used for Reynolds-averaged simulations of scramjet isolator flows. The turbulence models considered were the Menter BSL, Menter SST, Wilcox 1998, Wilcox 2006, and the Gatski-Speziale explicit algebraic Reynolds stress models. The simulations were carried out using the VULCAN computational fluid dynamics package developed at the NASA Langley Research Center. A procedure to quantify the numerical errors was developed to account for discretization errors in the validation process. This procedure utilized the grid convergence index defined by Roache as a bounding estimate for the numerical error. The validation data was collected from a mechanically back-pressured constant area (1 2 inch) isolator model with an isolator entrance Mach number of 2.5. As expected, the model-form uncertainty was substantial for the shock-dominated, massively separated flowfield within the isolator as evidenced by a 6 duct height variation in shock train length depending on the turbulence model employed. Generally speaking, the turbulence models that did not include an explicit stress limiter more closely matched the measured surface pressures. This observation is somewhat surprising, given that stress-limiting models have generally been developed to better predict shock-separated flows. All of the models considered also failed to properly predict the shape and extent of the separated flow region caused by the shock boundary layer interactions. However, the best performing models were able to predict the isolator shock train length (an important metric for isolator operability margin) to within 1 isolator duct height.

Baurle, Robert A.↗
NASA NTRS2012

Development of an OSSE Framework for a Global Atmospheric Data Assimilation System

Observing system simulation experiments (OSSEs) are powerful tools for estimating the usefulness of various configurations of envisioned observing systems and data assimilation techniques. Their utility stems from their being conducted in an entirely simulated context, utilizing simulated observations having simulated errors and drawn from a simulation of the earth's environment. Observations are generated by applying physically based algorithms to the simulated state, such as performed during data assimilation or using other appropriate algorithms. Adding realistic instrument plus representativeness errors, including their biases and correlations, can be critical for obtaining realistic assessments of the impact of a proposed observing system or analysis technique. If estimates of the expected accuracy of proposed observations are realistic, then the OSSE can be also used to learn how best to utilize the new information, accelerating its transition to operations once the real data are available. As with any inferences from simulations, however, it is first imperative that some baseline OSSEs are performed and well validated against corresponding results obtained with a real observing system. This talk provides an overview of, and highlights critical issues related to, the development of an OSSE framework for the tropospheric weather prediction component of the NASA GEOS-5 global atmospheric data assimilation system. The framework includes all existing observations having significant impact on short-term forecast skill. Its validity has been carefully assessed using a range of metrics that can be evaluated in both the OSSE and real contexts, including adjoint-based estimates of observation impact. A preliminary application to the Aeolus Doppler wind lidar mission, scheduled for launch by the European Space Agency in 2014, has also been investigated.

Gelaro, Ronald↗
NASA NTRS2013

A Signal Detection Theory Approach to Evaluating Oculometer Data Quality

Currently, data quality is described in terms of spatial and temporal accuracy and precision [Holmqvist et al. in press]. While this approach provides precise errors in pixels, or visual angle, often experiments are more concerned with whether subjects'points of gaze can be said to be reliable with respect to experimentally-relevant areas of interest. This paper proposes a method to characterize oculometer data quality using Signal Detection Theory (SDT) [Marcum 1947]. SDT classification results in four cases: Hit (correct report of a signal), Miss (failure to report a ), False Alarm (a signal falsely reported), Correct Reject (absence of a signal correctly reported). A technique is proposed where subjects' are directed to look at points in and outside of an AOI, and the resulting Points of Gaze (POG) are classified as Hits (points known to be internal to an AOI are classified as such), Misses (AOI points are not indicated as such), False Alarms (points external to AOIs are indicated as in the AOI), or Correct Rejects (points external to the AOI are indicated as such). SDT metrics describe performance in terms of discriminability, sensitivity, and specificity. This paper presentation will provide the procedure for conducting this assessment and an example of data collected for AOIs in a simulated flightdeck environment.

Latorella, Kara↗
NASA NTRS2019

Large Contribution of Meteorological Factors to Inter-Decadal Changes in Regional Aerosol Optical Depth

Aerosol optical depth (AOD) has become a crucial metric for assessing global climate change. Although global and regional AOD trends have been studied extensively, it remains unclear what factors are driving the inter-decadal variations in regional AOD and how to quantify the relative contribution of each dominant factor. This study used a long-term (1980–2016) aerosol dataset from the Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2) reanalysis, along with two satellite-based AOD datasets (MODIS/Terra and MISR) from 2001 to 2016, to investigate the long-term trends in global and regional aerosol loading. Statistical models based on emission factors and meteorological parameters were developed to identify the main factors driving the inter-decadal changes of regional AOD and to quantify their contribution. Evaluation of the MERRA-2 AOD with the ground-based measurements of AERONET indicated significant spatial agreement on the global scale (r= 0.85, root-mean-square error = 0.12, mean fractional error = 38.7 %, fractional gross error = 9.86 % and index of agreement = 0.94). However, when AOD observations from the China Aerosol Remote Sensing Network (CARSNET) were employed for independent verification, the results showed that MERRA-2 AODs generally underestimated CARSNET AODs in China (relative mean bias = 0.72 and fractional gross error =−34.3 %). In general, MERRA-2 was able to quantitatively reproduce the annual and seasonal AOD trends on both regional and global scales, as observed by MODIS/Terra, although some differences were found when compared to MISR. Over the 37-year period in this study, significant decreasing trends were observed over Europe and the eastern United States. In contrast, eastern China and southern Asia showed AOD increases, but the increasing trend of the former reversed sharply in the most recent decade. The statistical analyses suggested that the meteorological parameters explained a larger proportion of the AOD variability (20.4 %–72.8 %) over almost all regions of interest (ROIs) during 1980–2014 when compared with emission factors (0 %–56 %). Further analysis also showed that SO2 was the dominant emission factor, explaining 12.7 %–32.6 % of the variation in AOD over anthropogenic-aerosol-dominant regions, while black carbon or organic carbon was the leading factor over the biomass-burning-dominant (BBD) regions, contributing 24.0 %–27.7 % of the variation. Additionally, wind speed was found to be the leading meteorological parameter, explaining 11.8 %–30.3 % of the variance over the mineral-dust-dominant regions, while ambient humidity (including soil moisture and relative humidity) was the top meteorological parameter over the BBD regions, accounting for 11.7 %–35.5 % of the variation. The results of this study indicate that the variation in meteorological parameters is a key factor in determining the inter-decadal change in regional AOD.

Huizheng Che↗
NASA NTRSDate not supplied

Leveraging Human Performance Data to Change the Narrative that People are the Safety Problem

The study of errors and failure has a long and productive history in the behavioral sciences. By studying how systems fail, we rule out various mechanisms for how those systems might work, thereby refining our theories of how they actually work. Human performance, however, includes more than errors; human performance comprises both failures and successes. A systematic bias to collect and analyze data only on error affects the decisions we make as a community by promoting the narrative that “people are the safety problem.” This narrative manifests in both obvious and subtle ways in the design of systems intended for human use. When the only safety data that are available are about human failure, then “data-driven” designs can only consider that humans fail. Changing this narrative will depend on new data and new ways to examine data – specifically, data on the processes by which human create and contribute to safety. An alternate narrative is that people represent a primary source of safety, through their capability to anticipate, monitor for, respond to, and learn from expected and unexpected change. This presentation will describe research efforts to expand the range of safety-relevant events to include not just rare safety failures but frequent safety successes. These efforts include use of data from both operations and simulations to develop methods and metrics for learning from structured observation, self-report, and system data.

Jon Holbrook↗
NASA NTRS1994

Spacecraft-spacecraft radio-metric tracking: Signal acquisition requirements and application to Mars approach navigation

Doppler and ranging measurements between spacecraft can be obtained only when the ratio of the total received signal power to noise power density (P(sub t)/N(sub 0)) at the receiving spacecraft is sufficiently large that reliable signal detection can be achieved within a reasonable time period. In this article, the requirement on P(sub t)/N(sub 0) for reliable carrier signal detection is calculated as a function of various system parameters, including characteristics of the spacecraft computing hardware and a priori uncertainty in spacecraft-spacecraft relative velocity and acceleration. Also calculated is the P(sub t)/N(sub 0) requirements for reliable detection of a ranging signal, consisting of a carrier with pseudonoise (PN) phase modulation. Once the P(sub t)/N(sub 0) requirement is determined, then for a given set of assumed spacecraft telecommunication characteristics (transmitted signal power, antenna gains, and receiver noise temperatures) it is possible to calculate the maximum range at which a carrier signal or ranging signal may be acquired. For example, if a Mars lander and a spacecraft approaching Mars are each equipped with 1-m-diameter antennas, the transmitted power is 5 W, and the receiver noise temperatures are 350 K, then S-band carrier signal acquisition can be achieved at ranges exceeding 10 million km. An error covariance analysis illustrates the utility of in situ Doppler and ranging measurements for Mars approach navigation. Covariance analysis results indicate that navigation accuracies of a few km can be achieved with either data type. The analysis also illustrates dependency of the achievable accuracy on the approach trajectory velocity.

Kahn, R. D.↗
NASA NTRS2021

Satellites for long-term monitoring of inland U.S. lakes: The MERIS time series and application for chlorophyll-a

Lakes and other surface fresh waterbodies provide drinking water, recreational and economic opportunities, food, and other critical support for humans, aquatic life, and ecosystem health. Lakes are also productive ecosystems that provide habitats and influence global cycles. Chlorophyll concentration provides a common metric of water quality, and is frequently used as a proxy for lake trophic state. Here, we document the generation and distribution of the complete MEdium Resolution Imaging Spectrometer (MERIS; Appendix A provides a complete list of abbreviations) radiometric time series for over 2300 satellite resolvable inland bodies of water across the contiguous United States (CONUS) and more than 5,000 in Alaska. This contribution greatly increases the ease of use of satellite remote sensing data for inland water quality monitoring, as well as highlights new horizons in inland water remote sensing algorithm development. We evaluate the performance of satellite remote sensing Cyanobacteria Index (CI)-based chlorophyll algorithms, the retrievals for which provide surrogate estimates of phytoplankton concentrations in cyanobacteria dominated lakes. Our analysis quantifies the algorithms' abilities to assess lake trophic state across the CONUS. As a case study, we apply a bootstrapping approach to derive a new CI-to-chlorophyll relationship, ChlBS, which performs relatively well with a multiplicative bias of 1.11 (11%) and mean absolute error of 1.60 (60%). While the primary contribution of this work is the distribution of the MERIS radiometric timeseries, we provide this case study as a roadmap for future stakeholders' algorithm development activities, as well as a tool to assess the strengths and weaknesses of applying a single algorithm across CONUS.

MERIS timeseries↗
NASA NTRS2009

Trading Robustness Requirements in Mars Entry Trajectory Design

One of the most important metrics characterizing an atmospheric entry trajectory in preliminary design is the size of its predicted landing ellipse. Often, requirements for this ellipse are set early in design and significantly influence both the expected scientific return from a particular mission and the cost of development. Requirements typically specify a certain probability level (6-level) for the prescribed ellipse, and frequently this latter requirement is taken at 36. However, searches for the justification of 36 as a robustness requirement suggest it is an empirical rule of thumb borrowed from non-aerospace fields. This paper presents an investigation into the sensitivity of trajectory performance to varying robustness (6-level) requirements. The treatment of robustness as a distinct objective is discussed, and an analysis framework is presented involving the manipulation of design variables to effect trades between performance and robustness objectives. The scenario for which this method is illustrated is the ballistic entry of an MSL-class Mars entry vehicle. Here, the design variable is entry flight path angle, and objectives are parachute deploy altitude performance and error ellipse robustness. Resulting plots show the sensitivities between these objectives and trends in the entry flight path angles required to design to these objectives. Relevance to the trajectory designer is discussed, as are potential steps for further development and use of this type of analysis.

Lafleur, Jarret M.↗
NASA NTRS2017

Novel Estimation of Pilot Performance Characteristics

Two mechanisms internal to the pilot that affect performance during a tracking task are: 1) Pilot equalization (i.e. lead/lag); and 2) Pilot gain (i.e. sensitivity to the error signal). For some applications McRuer's Crossover Model can be used to anticipate what equalization will be employed to control a vehicle's dynamics. McRuer also established approximate time delays associated with different types of equalization - the more cognitive processing that is required due to equalization difficulty, the larger the time delay. However, the Crossover Model does not predict what the pilot gain will be. A nonlinear pilot control technique, observed and coined by the authors as 'amplitude clipping', is shown to improve stability, performance, and reduce workload when employed with vehicle dynamics that require high lead compensation by the pilot. Combining linear and nonlinear methods a novel approach is used to measure the pilot control parameters when amplitude clipping is present, allowing precise measurement in real time of key pilot control parameters. Based on the results of an experiment which was designed to probe workload primary drivers, a method is developed that estimates pilot spare capacity from readily observable measures and is tested for generality using multi-axis flight data. This paper documents the initial steps to developing a novel, simple objective metric for assessing pilot workload and its variation over time across a wide variety of tasks. Additionally, it offers a tangible, easily implementable methodology for anticipating a pilot's operating parameters and workload, and an effective design tool. The model shows promise in being able to precisely predict the actual pilot settings and workload, and observed tolerance of pilot parameter variation over the course of operation. Finally, an approach is proposed for generating Cooper-Harper ratings based on the workload and parameter estimation methodology.

Human pilot models↗
NASA NTRS2005

Tools of the Future: How Decision Tree Analysis Will Impact Mission Planning

The universe is infinitely complex; however, the human mind has a finite capacity. The multitude of possible variables, metrics, and procedures in mission planning are far too many to address exhaustively. This is unfortunate because, in general, considering more possibilities leads to more accurate and more powerful results. To compensate, we can get more insightful results by employing our greatest tool, the computer. The power of the computer will be utilized through a technology that considers every possibility, decision tree analysis. Although decision trees have been used in many other fields, this is innovative for space mission planning. Because this is a new strategy, no existing software is able to completely accommodate all of the requirements. This was determined through extensive research and testing of current technologies. It was necessary to create original software, for which a short-term model was finished this summer. The model was built into Microsoft Excel to take advantage of the familiar graphical interface for user input, computation, and viewing output. Macros were written to automate the process of tree construction, optimization, and presentation. The results are useful and promising. If this tool is successfully implemented in mission planning, our reliance on old-fashioned heuristics, an error-prone shortcut for handling complexity, will be reduced. The computer algorithms involved in decision trees will revolutionize mission planning. The planning will be faster and smarter, leading to optimized missions with the potential for more valuable data.

Otterstatter, Matthew R.↗
NASA NTRSDate not supplied

Oversimplification of Systems Engineering Goals, Processes, and Criteria in NASA Space Life Support

This paper investigates the oversimplification of the inherently complex systems engineering process in space life support. The standard systems engineering process steps are described. The International Space Station (ISS) life support system is explained with its goals and performance criteria. Although it is not usually emphasized, the essential function of developing a hierarchy of systems and subsystems is to simplify the design process. The System Complexity Metric (SCM) shows how this di-vide-and-conquer approach also reduces the system complexity. The complete systems engineering process has many detailed steps. It is often simplified because of the effort required and the human limitations on working memory and decision span. Systems analysis demands slow, logical, and fo-cused thinking but is often bypassed in favor of quick, intuitive, subconscious “gut feel.” A study of 100 system designs found examples of 12 specific mental mistakes, such as ignoring stakeholder needs, and these mistakes are essentially oversimplifications of the systems engineering process. An analysis of space life support goals, options, criteria, and processes found 11 examples of oversimplifications in systems engineering, such as neglecting safety and cost. All these 11 oversimplifications could be traced to one or more of the 12 previously identified mental mistakes or other well-known ones, such as ig-noring sunk costs. Oversimplification of the systems engineering process is rarely noticed but is a common and harmful problem. A study of failures in 50 different space systems found that problems in systems engineering caused failures and often led to errors in design, development, and test that further contributed to failure. It seems that more diligent systems engineering could prevent many project problems and failures, but projects seem to be more guided by “gut feel” based on tradition, authority, and consensus than on the logical, rational systems engineering approach.

Simplified systems engineering↗
NASA NTRS1993

Determination of use of a real time tone tracker to obtain same beam interferometry data

The radio metric tracking technique known as Same-Beam Interferometry (SBI) has been shown to improve orbit determination accuracy for the Magellan and Pioneer 12 orbiter. Previous efforts to explore the technique were carried out by making open loop recordings of the carrier signals from the two spacecraft and extracting their phases through post processing. This paper reports on the use of a closed loop receiver to simultaneously measure the carrier signals from two spacecraft in order to produce SBI data in near real time. The Experiment Tone Tracker is a digital closed loop receiver installed in two of NASA's Deep Space Network stations which can simultaneously extract the phase of up to eight tones. The receivers were used in late September and October of 1992 to collect Doppler and SBI data from Pioneer 12 and Magellan. The demise of the Pionner 12 on October 8th during the start-up phase of our tests precluded the collection of an extensive set of SBI data, however two passes of SBI and several arcs of single spacecraft Doppler data were recorded. The SBI data were analyzed and determined to have statistical errors consistent with error models and similar to open loop data.

Nandi, S.↗
NASA NTRS2022

Uncertainty Models for the Hybrid Parametric Variation Method of Uncertainty Quantification; Analysis

There is some level of uncertainty in every finite element model (FEM), which flows to a level of uncertainty in predicted results. The purpose of uncertainty quantification (UQ) is to provide statistical bounds on prediction accuracy based on model uncertainty. This is distinct from model updating, which attempts to modify models to improve their accuracy. UQ does not improve the accuracy of models, but accepts that the models are inherently inaccurate and attempts to quantify the impact of that inaccuracy on predicted results. Previously, an alternate method for UQ, called the Hybrid Parametric Variation (HPV) method, was applied to Space Launch System (SLS) Hurty/Craig-Bampton (HCB) components to predict system-level statistics for launch vehicle attitude control transfer functions and core stage section loads due to buffet. The HPV method combines a parametric variation of the HCB fixed-interface (FI) modal frequencies with a nonparametric variation (NPV) method that randomly varies the HCB mass and stiffness matrices as Wishart random matrix distributions using random matrix theory (RMT). Alternatively, the most common method for modeling uncertainty in the structural dynamics community is a parametric approach, which varies physical parameters in the model. However, there are several disadvantages associated with the parametric method. Determining a reduced set of parameters that have a significant impact on the system response can be time consuming, and the selected parameter probability distributions are rarely reliably known. Therefore, in practice, the parameters are surrogates for the actual errors, and the link to parameter uncertainty is unknown. Another major drawback is that the uncertainty that can be represented is limited to the form of the nominal FEM. It is the experience of the authors that based on numerous aerospace programs, almost all FEM errors are in form rather than parameter values. This hypothesis is supported by the observation of the authors that it is almost never possible to ‘tune’ a FEM to match modal test results by only modifying model parameters. Model-form uncertainty cannot be directly represented by FEM input parameters nor included in a parametric approach. However, model-form uncertainty can be modeled using RMT, where a probability distribution is developed for the matrix ensemble of interest. The major advantage of the NPV method is that it covers errors in model form. The HPV method anchors uncertainty at the HCB component level to component modal test results by matching the HCB and test modes based on mode descriptions or other methods, and then applying differing levels of frequency variation. The specific variations depend on the confidence to which a component FEM has been validated through modal testing. The NPV method is layered on the frequency variation to match modal test self-orthogonality and cross-orthogonality (XO) results. Once the component uncertainty models are identified, they are assembled, and the uncertainty is propagated to the system level using a Monte Carlo (MC) analysis approach that generates statistics for system-level predictions This provides a UQ method that can be traced to test data, which can be updated as additional data and improved correlated models become available. The purpose of this paper is to collect and present all of the theory for HPV that has been previously published in reports and papers and to present examples of its application. Specifically, component uncertainty models based on the dispersion of corresponding mass and stiffness matrices using proposed test/analysis correlation metrics are investigated. The first example is purely academic so that the true answers are known, and the validity of the HPV method and the corresponding uncertainty models can be determined. The purpose of this paper is to collect and present all of the theory for HPV that has been previously published in reports and papers and to present examples of its application. Specifically, component uncertainty models based on the dispersion of corresponding mass and stiffness matrices using proposed test/analysis correlation metrics are investigated. The first example is purely academic so that the true answers are known, and the validity of the HPV method and the corresponding uncertainty models can be determined. The second example is an application to a component that is design specific to the SLS. Based on this work and other assessments, the HPV method provides another tool to the toolset used for complex system UQ analysis. From experience gathered to date using the HPV method, additional design specific applications must be investigated to provide further confidence in the validity of the HPV method of UQ analysis.

Uncertainty quantification↗