Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

The DeepMIP Contribution to PMIP4: Experimental Design for Model Simulations of the EECO, PETM, and pre-PETM (version 1.0)

Past warm periods provide an opportunity to evaluate climate models under extreme forcing scenarios, in particular high ( greater than 800 ppmv) atmospheric CO2 concentrations. Although a post hoc intercomparison of Eocene (approximately 50 Ma) climate model simulations and geological data has been carried out previously, models of past high-CO2 periods have never been evaluated in a consistent framework. Here, we present an experimental design for climate model simulations of three warm periods within the early Eocene and the latest Paleocene (the EECO, PETM, and pre-PETM). Together with the CMIP6 pre-industrial control and abrupt 4(times) CO2 simulations, and additional sensitivity studies, these form the first phase of DeepMIP - the Deep-time Model Intercomparison Project, itself a group within the wider Paleoclimate Modeling Intercomparison Project (PMIP). The experimental design specifies and provides guidance on boundary conditions associated with palaeogeography, greenhouse gases, astronomical configuration, solar constant, land surface processes, and aerosols. Initial conditions, simulation length, and output variables are also specified. Finally, we explain how the geological data sets, which will be used to evaluate the simulations, will be developed.

climate models↗
NASA NTRS2000

First-Order Frameworks for Managing Models in Engineering Optimization

Approximation/model management optimization (AMMO) is a rigorous methodology for attaining solutions of high-fidelity optimization problems with minimal expense in high- fidelity function and derivative evaluation. First-order AMMO frameworks allow for a wide variety of models and underlying optimization algorithms. Recent demonstrations with aerodynamic optimization achieved three-fold savings in terms of high- fidelity function and derivative evaluation in the case of variable-resolution models and five-fold savings in the case of variable-fidelity physics models. The savings are problem dependent but certain trends are beginning to emerge. We give an overview of the first-order frameworks, current computational results, and an idea of the scope of the first-order framework applicability.

Alexandrov, Natlia M.↗
NASA NTRS1973

Significant techniques in the processing and interpretation of ERTS-1 data

The discipline oriented investigations underway at the Johnson Space Center (JSC) using ERTS-1 data provide an appropriate framework for the systematic evaluation of the various elements comprising a prototype multispectral data processing and analysis system. In particular such a system may be thought of as the integration of: (1) a preprocessing subsystem; (2) a spectral clustering subsystem, (3) a correlation and classification subsystem; (4) mensuration subsystem; and (5) an information management subsystem. Specific elements of this system are already operational at JSC. It is in the context of this system that technique development and application is being pursued at JSC. Aircraft, ERTS and EREP data will be utilized to refine the subsystem elements for each of the data acquisition systems or system combinations that are optimally suited for a specific Earth Resources application. The techniques reported are those that have been developed to date during the utilization of ERTS-1 data in this processing and analysis system.

Cousin, S. B.↗
NASA NTRS1983

IGCC performance comparison for variations in gasifier type and gas turbine firing temperature

Performance estimates were made for a series of integrated coal gasification combined cycle (IGCC) power systems using three generic types of coal gasification subsystems. The objectives of this study were (1) to provide a self consistent comparison of IGCC systems using different types of gasifiers and different oxidants and (2) to use this framework of cases to evaluate the effect of a gas turbine firing temperature and cooling approach an overall system efficiency. The basic IGCC systems considered included both air and oxygen blown versions of a fluidized bed gasifier, represented by the Westinghouse design, and an entrained bed gasifier, represented by the Texaco design. Also considered were systems using an oxygen blown, fixed bed gasifier, represented by the British Gas Corporation (BGC) slagging gasifier. All of these gasifiers were integrated with a combined cycle using a gas turbine firing temperature of 1700 K (2600 F) and a compressor pressure ratio of 16:1. Steam turbine throttle conditions were chosen to be 16.6 MPa/811 K (2400 psia/1000 F) with a single reheat to 810 K (1000 F). Some of these cases were modified to allow the evaluation of the effect of gas turbine firing temperature. Turbine firing temperatures from state of the art 1365 K (2000 F) to an advanced technology 1920 K (3000 F) were analyzed. A turbine cooling technology that maintains metal temperatures below acceptable limits was assumed for each level of firing temperature. System performance comparisons were made using three advanced turbine cooling technologies for the 1920 K (3000 F) firing temperature. The results indicate that the IGCC using the BGC gasifier had the highest net system efficiency (42.1 percent) of the five gasification cases considered.

Stochl, R. J.↗
NASA NTRS1992

Knowledge modeling for software design

This paper develops a modeling framework for systems engineering that encompasses systems modeling, task modeling, and knowledge modeling, and allows knowledge engineering and software engineering to be seen as part of a unified developmental process. This framework is used to evaluate what novel contributions the 'knowledge engineering' paradigm has made and how these impact software engineering.

Shaw, Mildred L. G.↗
NASA NTRS2003

A Comprehension Based Analysis of Autoflight System Interfaces

This cooperative agreement supported Dr. Peter Polson's participation in two interrelated research programs. The first was the development of the Situation-Goal-Behavior (SGB) Model that is both a formal description of an avionics system's logic and behavior and a representation of a system that can be understood by avionics designers, pilots, and training developers. The second was the development of a usability inspection method based on an approximate model, RAFIV, of pilot interactions with the Flight Management System (FMS). The main purpose of this report is to integrate the two models and provide a context in order to better characterize the accomplishments of this research program. A major focus of both the previous and this Cooperative Agreement was the development of usability evaluation methods that can be effectively utilized during all phases of the design, development, and certification process of modern avionics systems. The current efforts to validate these methods have involved showing that they generate useful analyses of known operational and training problems with the current generation of avionics systems in modern commercial airliners. This report is organized into seven sections. Following the overview, the second section describes the Goal-Situation-Behavior model and its applications. The next section summarizes the foundations of the RAFIV model and describes the model in some detail. The contents of both these sections are derived from previous reports referenced in footnotes. The fourth section integrates these two models into a complete design evaluation and training development framework. The fifth section contains conclusions and possible future directions for research. References are in Section 6. Section 7 contains the titles and abstracts of the papers paper describing in more detail the results of this research program.

Palmer, Everett↗
NASA NTRS2003

Resource Management for Real-Time Adaptive Agents

Increased autonomy and automation in onboard flight systems offer numerous potential benefits, including cost reduction and greater flexibility. The existence of generic mechanisms for automation is critical for handling unanticipated science events and anomalies where limitations in traditional control software with fixed, predetermined algorithms can mean loss of science data and missed opportunities for observing important terrestrial events. We have developed such a mechanism by adding a Hierarchical Agent-based ReaLTime technology (HART) extension to our Dynamic Resource Management (DRM) middleware. Traditional DRM provides mechanisms to monitor the realtime performance of distributed applications and to move applications among processors to improve real-time performance. In the HART project we have designed and implemented a performance adaptation mechanism to improve reaktime performance. To use this mechanism, applications are developed that can run at various levels of quality. The DRM can choose a setting for the quality level of an application dynamically at run-time in order to manage satellite resource usage more effectively. A groundbased prototype of a satellite system that captures and processes images has also been developed as part of this project to be used as a benchmark for evaluating the resource management framework A significant enhancement of this generic mission-independent framework allows scientists to specify the utility, or "scientific benefit," of science observations under various conditions like cloud cover and compression method. The resource manager then uses these benefit tables to determine in redtime how to set the quality levels for applications to maximize overall system utility as defined by the scientists running the mission. We also show how maintenance functions llke health and safety data can be integrated into the utility framework. Once thls framework has been certified for missions and successfully flight tested it can be reused with little development overhead for other missions. In contrast, current space missions llke Swift manage similar types of resource trade -off completely with the scientific application code itself, and such code must be re-certified and tested for each mission even if a large portion of the code base is shared. This final report discusses some of the major issues motivating this research effort, provides a literature review of the related work, discusses the resource management framework and ground-based satellite system prototype that has been developed, indicates what work is yet to be performed, and provides a list of publications resulting from this work.

Welch, Lonnie↗
NASA NTRS2007

Robustness Analysis of Integrated LPV-FDI Filters and LTI-FTC System for a Transport Aircraft

This paper proposes an analysis framework for robustness analysis of a nonlinear dynamics system that can be represented by a polynomial linear parameter varying (PLPV) system with constant bounded uncertainty. The proposed analysis framework contains three key tools: 1) a function substitution method which can convert a nonlinear system in polynomial form into a PLPV system, 2) a matrix-based linear fractional transformation (LFT) modeling approach, which can convert a PLPV system into an LFT system with the delta block that includes key uncertainty and scheduling parameters, 3) micro-analysis, which is a well known robust analysis tool for linear systems. The proposed analysis framework is applied to evaluating the performance of the LPV-fault detection and isolation (FDI) filters of the closed-loop system of a transport aircraft in the presence of unmodeled actuator dynamics and sensor gain uncertainty. The robustness analysis results are compared with nonlinear time simulations.

Khong, Thuan H.↗
NASA NTRS2009

Planetary Geologic Mapping Handbook - 2009

Geologic maps present, in an historical context, fundamental syntheses of interpretations of the materials, landforms, structures, and processes that characterize planetary surfaces and shallow subsurfaces (e.g., Varnes, 1974). Such maps also provide a contextual framework for summarizing and evaluating thematic research for a given region or body. In planetary exploration, for example, geologic maps are used for specialized investigations such as targeting regions of interest for data collection and for characterizing sites for landed missions. Whereas most modern terrestrial geologic maps are constructed from regional views provided by remote sensing data and supplemented in detail by field-based observations and measurements, planetary maps have been largely based on analyses of orbital photography. For planetary bodies in particular, geologic maps commonly represent a snapshot of a surface, because they are based on available information at a time when new data are still being acquired. Thus the field of planetary geologic mapping has been evolving rapidly to embrace the use of new data and modern technology and to accommodate the growing needs of planetary exploration. Planetary geologic maps have been published by the U.S. Geological Survey (USGS) since 1962 (Hackman, 1962). Over this time, numerous maps of several planetary bodies have been prepared at a variety of scales and projections using the best available image and topographic bases. Early geologic map bases commonly consisted of hand-mosaicked photographs or airbrushed shaded-relief views and geologic linework was manually drafted using mylar bases and ink drafting pens. Map publishing required a tedious process of scribing, color peel-coat preparation, typesetting, and photo-laboratory work. Beginning in the 1990s, inexpensive computing, display capability and user-friendly illustration software allowed maps to be drawn using digital tools rather than pen and ink, and mylar bases became obsolete. Terrestrial geologic maps published by the USGS now are primarily digital products using geographic information system (GIS) software and file formats. GIS mapping tools permit easy spatial comparison, generation, importation, manipulation, and analysis of multiple raster image, gridded, and vector data sets. GIS software has also permitted the development of project-specific tools and the sharing of geospatial products among researchers. GIS approaches are now being used in planetary geologic mapping as well (e.g., Hare and others, 2009). Guidelines or handbooks on techniques in planetary geologic mapping have been developed periodically (e.g., Wilhelms, 1972, 1990; Tanaka and others, 1994). As records of the heritage of mapping methods and data, these remain extremely useful guides. However, many of the fundamental aspects of earlier mapping handbooks have evolved significantly, and a comprehensive review of currently accepted mapping methodologies is now warranted. As documented in this handbook, such a review incorporates additional guidelines developed in recent years for planetary geologic mapping by the NASA Planetary Geology and Geophysics (PGG) Program s Planetary Cartography and Geologic Mapping Working Group s (PCGMWG) Geologic Mapping Subcommittee (GEMS) on the selection and use of map bases as well as map preparation, review, publication, and distribution. In light of the current boom in planetary exploration and the ongoing rapid evolution of available data for planetary mapping, this handbook is especially timely.

Tanaka, K. L.↗
NASA NTRS2010

Planetary Geologic Mapping Handbook - 2010

Geologic maps present, in an historical context, fundamental syntheses of interpretations of the materials, landforms, structures, and processes that characterize planetary surfaces and shallow subsurfaces. Such maps also provide a contextual framework for summarizing and evaluating thematic research for a given region or body. In planetary exploration, for example, geologic maps are used for specialized investigations such as targeting regions of interest for data collection and for characterizing sites for landed missions. Whereas most modern terrestrial geologic maps are constructed from regional views provided by remote sensing data and supplemented in detail by field-based observations and measurements, planetary maps have been largely based on analyses of orbital photography. For planetary bodies in particular, geologic maps commonly represent a snapshot of a surface, because they are based on available information at a time when new data are still being acquired. Thus the field of planetary geologic mapping has been evolving rapidly to embrace the use of new data and modern technology and to accommodate the growing needs of planetary exploration. Planetary geologic maps have been published by the U.S. Geological Survey (USGS) since 1962. Over this time, numerous maps of several planetary bodies have been prepared at a variety of scales and projections using the best available image and topographic bases. Early geologic map bases commonly consisted of hand-mosaicked photographs or airbrushed shaded-relief views and geologic linework was manually drafted using mylar bases and ink drafting pens. Map publishing required a tedious process of scribing, color peel-coat preparation, typesetting, and photo-laboratory work. Beginning in the 1990s, inexpensive computing, display capability and user-friendly illustration software allowed maps to be drawn using digital tools rather than pen and ink, and mylar bases became obsolete. Terrestrial geologic maps published by the USGS now are primarily digital products using geographic information system (GIS) software and file formats. GIS mapping tools permit easy spatial comparison, generation, importation, manipulation, and analysis of multiple raster image, gridded, and vector data sets. GIS software has also permitted the development of projectspecific tools and the sharing of geospatial products among researchers. GIS approaches are now being used in planetary geologic mapping as well. Guidelines or handbooks on techniques in planetary geologic mapping have been developed periodically. As records of the heritage of mapping methods and data, these remain extremely useful guides. However, many of the fundamental aspects of earlier mapping handbooks have evolved significantly, and a comprehensive review of currently accepted mapping methodologies is now warranted. As documented in this handbook, such a review incorporates additional guidelines developed in recent years for planetary geologic mapping by the NASA Planetary Geology and Geophysics (PGG) Program's Planetary Cartography and Geologic Mapping Working Group's (PCGMWG) Geologic Mapping Subcommittee (GEMS) on the selection and use of map bases as well as map preparation, review, publication, and distribution. In light of the current boom in planetary exploration and the ongoing rapid evolution of available data for planetary mapping, this handbook is especially timely.

Tanaka, K. L.↗
NASA NTRS2010

Experimental Validation of L1 Adaptive Control: Rohrs' Counterexample in Flight

The paper presents new results on the verification and in-flight validation of an L1 adaptive flight control system, and proposes a general methodology for verification and validation of adaptive flight control algorithms. The proposed framework is based on Rohrs counterexample, a benchmark problem presented in the early 80s to show the limitations of adaptive controllers developed at that time. In this paper, the framework is used to evaluate the performance and robustness characteristics of an L1 adaptive control augmentation loop implemented onboard a small unmanned aerial vehicle. Hardware-in-the-loop simulations and flight test results confirm the ability of the L1 adaptive controller to maintain stability and predictable performance of the closed loop adaptive system in the presence of general (artificially injected) unmodeled dynamics. The results demonstrate the advantages of L1 adaptive control as a verifiable robust adaptive control architecture with the potential of reducing flight control design costs and facilitating the transition of adaptive control into advanced flight control systems.

Xargay, Enric↗
NASA NTRSDate not supplied

Protecting Crew and Surface Systems with a Long-Duration Lunar Safe Haven

NASA’s Artemis program will send astronauts to the lunar surface for extended mission durations throughout the 2030s, with a focus on sustainability and extensibility for Mars exploration, as described in NASA’s “Artemis Plan”. However, NASA must place more emphasis on protecting both the crew and the exploration surface systems if they hope to achieve long-duration sustainability on the lunar surface. It is now reasonably achievable with excavation, construction, and autonomy technologies to achieve a significant level of protection that architectures have been unable to achieve to date. The Lunar Safe Haven (LSH) was proposed to protect astronauts, electronics, and other surface exploration systems from the hazards of the lunar environment, including radiation, micrometeoroid strikes, lunar dust, thermal vacuum, etc. During the study, Level Zero Requirements were developed for the LSH, and a decision analysis framework was baselined to evaluate concepts. The LSH Study also performed a comprehensive trade study, during which it identified numerous alternatives for establishing and maintaining a safe haven shelter on the lunar surface. This paper reviews the products developed during the study and presents the final recommendations.

radiation shielding↗
NASA NTRSDate not supplied

PolarMERRA: A Polar-Focused Global Reanalysis Project for Scientific and Stakeholder Needs

The adequate modeling of physical processes in the Arctic and Antarctic is vital for developing prediction capabilities and for obtaining an understanding of rapidly evolving polar conditions, including their potential global impacts. These processes are often poorly represented in global models and reanalyses, owing to a legacy modeling focus on midlatitude processes, as well as a scarcity of observations needed for model development in polar regions. The polarMERRA initiative, a joint effort between NASA’s Cryospheric Sciences and Modeling and Prediction programs, seeks to improve the representation of cryospheric and polar atmospheric processes and to develop an open-source framework for a quantitively evaluation of polar-relevant variables against current and future satellite and in-situ observations, models, and reanalyses. Here, we evaluate the impacts of spatial resolution and modifications to sea ice, ice sheet, and atmospheric parameterizations on the representation of high latitude conditions by using a quantitative scorecard approach that leverages NASA’s extensive satellite record. Investigations are conducted using the NASA Goddard Earth Observing System model (GEOS) and its data assimilation system (GEOS DAS). Through a quantitative identification of process deficiencies, bias reductions in key surface variables, including temperature and precipitation over cryospheric surfaces, may be achieved. The polarMERRA project additionally seeks to identify additional data sources for use in the GEOS DAS, and to incorporate data and parameterization improvements into future model and reanalysis products for scientific and stakeholder use.

Lauren C Andrews↗
NASA NTRSDate not supplied

Process Qualification Concepts in NASA-STD-6030

- Terminology - Definition and role of QMP in the NASA-STD-6030 framework - Qualification checklist for evaluating a candidate AM material process - Types of data needed to evaluate the metrics for a QMP - Advocacy for a singular, contained data entity that enables QMP evaluation and other aspects of the AM ecosystem - Material Engineering Equivalence Baseline - Discussion of Material Engineering Equivalence concepts - Summary

Doug Wells↗
NASA NTRS1982

Models for evaluating the performability of degradable computing systems

Recent advances in multiprocessor technology established the need for unified methods to evaluate computing systems performance and reliability. In response to this modeling need, a general modeling framework that permits the modeling, analysis and evaluation of degradable computing systems is considered. Within this framework, several user oriented performance variables are identified and shown to be proper generalizations of the traditional notions of system performance and reliability. Furthermore, a time varying version of the model is developed to generalize the traditional fault tree reliability evaluation methods of phased missions.

Wu, L. T.↗
NASA NTRS2009

Benchmarking Diagnostic Algorithms on an Electrical Power System Testbed

Diagnostic algorithms (DAs) are key to enabling automated health management. These algorithms are designed to detect and isolate anomalies of either a component or the whole system based on observations received from sensors. In recent years a wide range of algorithms, both model-based and data-driven, have been developed to increase autonomy and improve system reliability and affordability. However, the lack of support to perform systematic benchmarking of these algorithms continues to create barriers for effective development and deployment of diagnostic technologies. In this paper, we present our efforts to benchmark a set of DAs on a common platform using a framework that was developed to evaluate and compare various performance metrics for diagnostic technologies. The diagnosed system is an electrical power system, namely the Advanced Diagnostics and Prognostics Testbed (ADAPT) developed and located at the NASA Ames Research Center. The paper presents the fundamentals of the benchmarking framework, the ADAPT system, description of faults and data sets, the metrics used for evaluation, and an in-depth analysis of benchmarking results obtained from testing ten diagnostic algorithms on the ADAPT electrical power system testbed.

Kurtoglu, Tolga↗
NASA NTRS2002

Statistical Analysis of CFD Solutions from the Drag Prediction Workshop

A simple, graphical framework is presented for robust statistical evaluation of results obtained from N-Version testing of a series of RANS CFD codes. The solutions were obtained by a variety of code developers and users for the June 2001 Drag Prediction Workshop sponsored by the AIAA Applied Aerodynamics Technical Committee. The aerodynamic configuration used for the computational tests is the DLR-F4 wing-body combination previously tested in several European wind tunnels and for which a previous N-Version test had been conducted. The statistical framework is used to evaluate code results for (1) a single cruise design point, (2) drag polars and (3) drag rise. The paper concludes with a discussion of the meaning of the results, especially with respect to predictability, Validation, and reporting of solutions.

Hemsch, Michael J.↗
NASA NTRS2022

Framework for Estimating Performance and Associated Uncertainty for Modified Aircraft Configurations

Flight testing has been the historical standard for determining aircraft airworthiness - however, increases in the cost of flight testing and the accuracy of inexpensive CFD promote certification by analysis to reduce or replace flight testing. A framework is introduced to predict the performance in the special case of a modification to an existing, previously certified aircraft. This framework uses a combination of existing flight test or high fidelity data of the original aircraft as well as lower fidelity data of the original and modified configurations. Two methods are presented which estimate the model form uncertainty of the modified configuration, which is then used to conduct non-deterministic simulations. The framework is applied to an example aircraft system with simulated flight test data to demonstrate the ability to predict the performance and associated uncertainty of modified aircraft configurations. However, it is important that the models and methods used are applicable and accurate throughout the intended use domain. The factors and limitations of the framework are explored to determine the range of applicability of the framework. The effects of these factors on the performance and uncertainty results are demonstrated using the example aircraft system. The framework is then applied to NASA’s X-57 Maxwell and each of its modifications. The estimated performance and associated uncertainties are then compared to the airworthiness criteria to evaluate the potential of the framework as a component to the certification by analysis process.

Uncertainty Quantification↗