Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Evaluation Framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS2011

Pilot/Controller Coordinated Decision Making in the Next Generation Air Transportation System

Introduction: NextGen technologies promise to provide considerable benefits in terms of enhancing operations and improving safety. However, there needs to be a thorough human factors evaluation of the way these systems will change the way in which pilot and controllers share information. The likely impact of these new technologies on pilot/controller coordinated decision making is considered in this paper using the "operational, informational and evaluative disconnect" framework. Method: Five participant focus groups were held. Participants were four experts in human factors, between x and x research students and a technical expert. The participant focus group evaluated five key NextGen technologies to identify issues that made different disconnects more or less likely. Results: Issues that were identified were: Decision Making will not necessarily improve because pilots and controllers possess the same information; Having a common information source does not mean pilots and controllers are looking at the same information; High levels of automation may lead to disconnects between the technology and pilots/controllers; Common information sources may become the definitive source for information; Overconfidence in the automation may lead to situations where appropriate breakdowns are not initiated. Discussion: The issues that were identified lead to recommendations that need to be considered in the development of NextGen technologies. The current state of development of these technologies provides a good opportunity to utilize recommendations at an early stage so that NextGen technologies do not lead to difficulties in resolving breakdowns in coordinated decision making.

Bearman, Chris↗
NASA NTRS2011

The First Flight Decision for New Human Spacecraft Vehicles - A General Approach

Determining when it is safe to fly a crew on a launch vehicle/spacecraft for the first time, especially when the test flight is a part of the overall system certification process, has long been a challenge for program decision makers. The decision on first flight is ultimately the judgment of the program and agency management in conjunction with the design and operations team. To aid in this decision process, a NASA team undertook the task to develop a generic framework for evaluating whether any given program or commercial provider has sufficiently complete and balanced plans in place to allow crewmembers to safely fly on human spaceflight systems for the first time. It was the team s goal to establish a generic framework that could easily be applied to any new system, although the system design and intended mission would require specific assessment. Historical data shows that there are multiple approaches that have been successful in first flight with crew. These approaches have always been tailored to the specific system design, mission objectives, and launch environment. Because specific approaches may vary significantly between different system designs and situations, prescriptive instructions or thorough checklists cannot be provided ahead of time. There are, however, certain general approaches that should be applied in thinking through the decision for first flight. This paper addresses some of the most important factors to consider when developing a new system or evaluating an existing system for whether or not it is safe to fly humans to/from space. In the simplest terms, it is time to fly crew for the first time when it is safe to do so and the benefit of the crewed flight is greater than the residual risk. This is rarely a straight-forward decision. The paper describes the need for experience, sound judgment, close involvement of the technical and management teams, and established decision processes. In addition, the underlying level of confidence the manager has in making the decision will also be discussed. By applying the outlined thought processes and approaches to a specific design, test program and mission objectives, a project team will be better able to focus the debate and discussion on critical areas for consideration and added scrutiny -- allowing decision makers to adequately address the first crewed flight decision.

Schaible, Dawn M.↗
NASA NTRSDate not supplied

New Solid Propellant Additives: Evaluation for NASA Mission Use Cases

Researchers and small companies have been investigating and developing new solid rocket motor propellant ingredients with renewed vigor. Lithium-aluminum alloy is an ingredient of particular interest, but most of the advertisement about its benefits are focused on defense tactical rocket applications. This paper develops a framework to evaluate this and other propellants against standard aluminized composite propellant for mass-limited missions like many NASA in-space applications. It shows a way to estimate density’s effect with an exponent less than 1 depending on mission and technology parameters, and an algorithm for directly calculating delta-velocity or payload performance improvement.

Lithium-aluminum↗
NASA NTRS2005

Diverse Applications of Occultation Data in Ozone Assimilation

Ozone profiles from solar occultation instruments provide invaluable information that can be used to evaluate the quality of assimilated ozone fields, from case studies to long-term phenomena. Occultation data can also be applied to include physical constraints while developing components of an assimilation system. Conversely, assimilation of occultation data can help in their validation, and it provides a framework for evaluation of the impact of occultation data on constraining global ozone fields within models. We illustrate these diverse applications by a series of examples using the ozone assimilation system at NASA/Goddard. In a case study, low ozone in the lower stratosphere due to transport of air from the Tropics to northern high latitudes that was captured by assimilation of Aura Microwave Limb Sounder (MLS) and Solar Backscatter Ultraviolet (SBUV) data, was found to agree with SAGE III data. For long-term monitoring, the quality of a multi-year SBUV-only assimilation was evaluated using monthly-mean time series of POAM, HALOE, and SAGE I1 data. We found realism in the representation of the annual cycle in ozone and in some aspects of interannual variability. Assimilation of POAM data was shown to improve the representation of lower stratospheric ozone, especially over Antarctica. More recently, we assimilated ILAS II ozone data in order to help in their validation. Solar occultation data are used to estimate parameters in a new model for forecast error variances that is being developed. These examples demonstrate the importance of occultation data for ozone assimilation, and potential of assimilation to increase the impact and the value of occultation data.

Markowitz, A.↗
NASA NTRSDate not supplied

pyQuARC: Open Source Library for Earth Observation Metadata Quality Assessment

Metadata quality is essential to effective data discovery and has become increasingly vital as more Earth Science data sets become available. The Common Metadata Repository (CMR) hosts metadata describing NASA’s Earth Observation data products, which are archived across 12 Distributed Active Archive Centers (DAACs). The Analysis and Review of CMR (ARC) Team, located at Marshall Space Flight Center, conducts metadata quality assessments to ensure that these data products are discoverable, accessible, and usable. To achieve these goals, the ARC team has developed a metadata quality assessment framework to evaluate metadata completeness, correctness, and consistency. ARC uses a combination of manual and automated methods to assess these three components and identify areas of improvement; the team then collaborates with the DAACs to resolve any findings. To streamline this process, ARC is currently developing a host of scripts, known as pyQuARC, to automate metadata quality assessments as much as possible. pyQuARC is an open source library for Earth Observation Metadata Quality Assessment, and the tool utilizes ARC’s metadata quality assessment framework to make basic validation checks, pinpoint inconsistencies between dataset-level (i.e. collection) and file-level (i.e. granule) metadata, and identify opportunities for more descriptive and robust information. Since pyQuARC is also customizable, other users can make modifications as needed, and future metadata standards can also be implemented. Once pyQuARC is fully developed, it will support multiple schema types to serve the broader EOSDIS metadata community. This presentation will provide an overview of pyQuARC and its process of development while showcasing the tool’s valuable features and uses.

Jenny Wood↗
NASA NTRS2019

Evaluation of NASA's High-Resolution Global Composition Simulations: Understanding a Pollution Event in the Chesapeake Bay During the Summer 2017 OWLETS Campaign

Recirculation of pollutants due to a bay breeze effect is a key meteorological mechanism impacting air quality near urban coastal areas, but regional and global chemical transport models have historically struggled to capture this phenomenon. We present a case study of a high ozone (O3) episode observed over the Chesapeake Bay during the NASA Ozone Water-Land Environmental Transition Study (OWLETS) in summer 2017. OWLETS included a complementary suite of ground-based and airborne observations, with which we characterize the meteorological and chemical context of this event and develop a framework to evaluate model performance. Two publicly-available NASA global high-resolution coupled chemistry-meteorology models (CCMMs) are investigated: GEOS-CF and MERRA2-GMI. The GEOS-CF R squared value for comparisons between the NASA Sherpa C-23 aircraft measurements to the GEOS-CF resulted in good agreement (R squared: 0.67) on July 19th and fair agreement (R squared: 0.55) for July 20th. Compared to surface observations, we find the GEOS-CF product with a 25 x 25 km squared grid box, at an hourly (R squared: 0.62 to 0.87) and 15-minute (R squared: 0.64 to 0.87) interval for six regional sites outperforms the hourly nominally 50 x 50 km squared gridded MERRA2-GMI (R squared: 0.53 to 0.76) for four of the six sites, suggesting it is better capable of simulating complex chemical and meteorological features associated with ozone transport within the Chesapeake Bay airshed. When the GEOS-CF product was compared to the TOLNet LiDAR observations at both NASA Langley Research Center (LaRC) and the Chesapeake Bay Bridge Tunnel (CBBT), the median differences at LaRC were -6 to 8% and at CBBT were ± 7% between 400 to 2000 m ASL. This indicates that, for this case study, the GEOS-CF is able to simulate surface level ozone diurnal cycles and vertical ozone profiles at small scales between the surface level and 2000 m ASL. Evaluating global chemical model simulations at sub-regional scales will help air quality scientists understand the complex processes occurring at small spatial and temporal scales within complex surface terrain changes, simulating nighttime chemistry and deposition, and the potential to use global chemical transport simulations in support of regional and sub-regional field campaigns.

NASA Ozone Water-Land Environmental Transition Stu↗
NASA NTRS2004

JACIE Radiometric Assessment of QuickBird Multispectral Imagery

Radiometric calibration of commercial imaging satellite products is required to ensure that science and application communities can place confidence in the imagery they use and can fully understand its properties. Inaccurate radiometric calibrations can lead to erroneous decisions and invalid conclusions and can limit intercomparisons with other systems. To address this calibration need, the NASA Stennis Space Center (SSC) Earth Science Applications (ESA) directorate,through the Joint Agency for Commercial Imagery Evaluation (JACIE) framework, established a commercial imaging satellite radiometric calibration team consisting of two groups: 1) NASA SSC ESA, supported by South Dakota State University, and 2) the University of Arizona Remote Sensing Group. The two groups determined the absolute radiometric calibration coefficients of the Digital Globe 4-band, 2.4-m QuickBird multispectral product covering the visible through near-infrared spectral region. For a 2-year period beginning in 2002, both groups employed some variant of a reflectance-based vicarious calibration approach, which required ground-based measurements coincident with QuickBird image acquisitions and radiative transfer calculations. The groups chose several study sites throughout the United States that covered nearly the entire dynamic range of the QuickBird sensor. QuickBird at-sensor radiance values were compared with those estimated by the two independent groups to determine the QuickBird sensor's radiometric accuracy. Approximately 20 at-sensor radiance estimates were vicariously determined each year. The estimates were combined to provide a high-precision radiometric gain calibration coefficient. The results of this evaluation provide the user community with an independent assessment of the QuickBird sensor's absolute calibration and stability over the 2-year period. While the techniques and method described reflect those developed at the NASA SSC, the results of both JACIE team groups are included in this paper.

Pagnutti, Mary↗
NASA NTRS2000

A simulator for evaluating methods for the detection of lesion-deficit associations

Although much has been learned about the functional organization of the human brain through lesion-deficit analysis, the variety of statistical and image-processing methods developed for this purpose precludes a closed-form analysis of the statistical power of these systems. Therefore, we developed a lesion-deficit simulator (LDS), which generates artificial subjects, each of which consists of a set of functional deficits, and a brain image with lesions; the deficits and lesions conform to predefined distributions. We used probability distributions to model the number, sizes, and spatial distribution of lesions, to model the structure-function associations, and to model registration error. We used the LDS to evaluate, as examples, the effects of the complexities and strengths of lesion-deficit associations, and of registration error, on the power of lesion-deficit analysis. We measured the numbers of recovered associations from these simulated data, as a function of the number of subjects analyzed, the strengths and number of associations in the statistical model, the number of structures associated with a particular function, and the prior probabilities of structures being abnormal. The number of subjects required to recover the simulated lesion-deficit associations was found to have an inverse relationship to the strength of associations, and to the smallest probability in the structure-function model. The number of structures associated with a particular function (i.e., the complexity of associations) had a much greater effect on the performance of the analysis method than did the total number of associations. We also found that registration error of 5 mm or less reduces the number of associations discovered by approximately 13% compared to perfect registration. The LDS provides a flexible framework for evaluating many aspects of lesion-deficit analysis.

NASA Program Biomedical Research and Countermeasur↗
NASA NTRS1973

Transport improvement, commuting costs, and residential location

A theoretical framework for evaluating the effects of introducing new transportation on residential travel patterns is presented. Data are based on changes in residential location of urban commuters that alter the mode and length of work trips as well as economic factors.

Stucker, J. P.↗
NASA NTRS1988

An examination of automation and robotics in the context of Space Station operations

A NASA-sponsored review of Space Station automation and robotics (A&R) applications from an operations and utilization perspective is presented. The goals of the A&R panel and this report are to identify major suggestions for advanced A&R operations application in Space Station as well as key technologies that have emerged or gained prominence since the completion of previous reports; to review and incorporate the range of possible Space Station A&R applications into a framework for evaluation of A&R opportunities; and to propose incentives for the government, work packages, and subcontractors to more aggressively identify, evaluate, and incorporate advanced A&R in Space Station Operations. The suggestions for A&R focused on narrow objectives using a conservative approach tuned to Space Station at IOC and limiting the Station's growth capabilities. A more aggressive stance is to identify functional needs over the Program's life, exploit and leverage available technology, and develop the key advanced technologies permitting effective use of A&R. The challenge is to systematically identify candidate functions to be automated, provide ways to create solutions resulting in savings or increased capabilities, and offer incentives that will promote the automation.

Criswell, David R.↗
NASA NTRS1993

Accuracy requirements

Satellite and surface measurements, if they are to serve as a climate monitoring system, must be accurate enough to permit detection of changes of climate parameters on decadal time scales. The accuracy requirements are difficult to define a priori since they depend on unknown future changes of climate forcings and feedbacks. As a framework for evaluation of candidate Climsat instruments and orbits, we estimate the accuracies that would be needed to measure changes expected over two decades based on theoretical considerations including GCM simulations and on observational evidence in cases where data are available for rates of change. One major climate forcing known with reasonable accuracy is that caused by the anthropogenic homogeneously mixed greenhouse gases (CO2, CFC's, CH4 and N2O). Their net forcing since the industrial revolution began is about 2 W/sq m and it is presently increasing at a rate of about 1 W/sq m per 20 years. Thus for a competing forcing or feedback to be important, it needs to be of the order of 0.25 W/sq m or larger on this time scale. The significance of most climate feedbacks depends on their sensitivity to temperature change. Therefore we begin with an estimate of decadal temperature change. Presented are the transient temperature trends simulated by the GISS GCM when subjected to various scenarios of trace gas concentration increases. Scenario B, which represents the most plausible near-term emission rates and includes intermittent forcing by volcanic aerosols, yields a global mean surface air temperature increase Delta Ts = 0.7 degrees C over the time period 1995-2015. This is consistent with the IPCC projection of about 0.3 degrees C/decade global warming (IPCC, 1990). Several of our estimates below are based on this assumed rate of warming.

Delgenio, Anthony↗
NASA NTRS1995

Microelectronics in Japan

The purpose of this JTEC study is to evaluate Japan's electronic manufacturing and packaging capabilities within the context of global economic competition. To carry out this study, the JTEC panel evaluated the framework of the Japanese consumer electronics industry and various technological and organizational factors that are likely to determine who will win and lose in the marketplace. This study begins with a brief overview of the electronics industry, especially as it operates in Japan today. Succeeding chapters examine the electronics infrastructure in Japan and take an in-depth look at the central issues of product development in order to identify those parameters that will determine future directions for electronic packaging technologies.

Boulton, William R.↗
NASA NTRS2002

Comparative Analysis of Radiometer Systems Using Non-Stationary Processes

Radiometers require periodic calibration to correct for instabilities in the receiver response. Various calibration techniques exist that minimize the effect of instabilities in the receivers. The optimal technique depends upon many parameters. Some parameters are constrained by the particular application and others can be chosen in the system design. For example, the measurement uncertainty may be reduced to the limits of the resolution of the measurement (sensitivity) if periodic absolute calibration can be performed with sufficient frequency. However if the period between calibrations is long, a reference-differencing technique, i.e. Dicke-type design, can yield better performance. The measurement uncertainty not only depends upon the detection scheme but also on the number of pixels between calibrations, the integration time per pixel, integration time per calibration reference measurement, calibration reference temperature, and the brightness temperature of what is being measured. The best scheme for reducing the measurement uncertainty also depends, in large part, on the stability of the receiver electronics. In this presentation a framework for evaluating calibration schemes for a wide range of system architectures is presented. Two methods for treating receiver non-stationarity are compared with radiometer measurements.

Racette, Paul↗
NASA NTRS2007

Entity-Centric Abstraction and Modeling Framework for Transportation Architectures

A comprehensive framework for representing transpportation architectures is presented. After discussing a series of preceding perspectives and formulations, the intellectual underpinning of the novel framework using an entity-centric abstraction of transportation is described. The entities include endogenous and exogenous factors and functional expressions are offered that relate these and their evolution. The end result is a Transportation Architecture Field which permits analysis of future concepts under the holistic perspective. A simulation model which stems from the framework is presented and exercised producing results which quantify improvements in air transportation due to advanced aircraft technologies. Finally, a modeling hypothesis and its accompanying criteria are proposed to test further use of the framework for evaluating new transportation solutions.

Lewe, Jung-Ho↗
NASA NTRS2011

Readiness for First Crewed Flight

The NASA Engineering and Safety Center (NESC) was requested to develop a generic framework for evaluating whether any given program has sufficiently complete and balanced plans in place to allow crewmembers to fly safely on a human spaceflight system for the first time (i.e., first crewed flight). The NESC assembled a small team which included experts with experience developing robotic and human spaceflight and aviation systems through first crewed test flight and into operational capability. The NESC team conducted a historical review of the steps leading up to the first crewed flights of Mercury through the Space Shuttle. Benchmarking was also conducted with the United States (U.S.) Air Force and U.S. Navy. This report contains documentation of that review.

Schaible, Dawn M.↗
NASA NTRS2012

LU60645GT and MA132843GT Catalogues of Lunar and Martian Impact Craters Developed Using a Crater Shape-based Interpolation Crater Detection Algorithm for Topography Data

For Mars, 57,633 craters from the manually assembled catalogues and 72,668 additional craters identified using several crater detection algorithms (CDAs) have been merged into the MA130301GT catalogue. By contrast, for the Moon the most complete previous catalogue contains only 14,923 craters. Two recent missions provided higher-quality digital elevation maps (DEMs): SELENE (in 1/16° resolution) and Lunar Reconnaissance Orbiter (we used up to 1/512°). This was the main motivation for work on the new Crater Shape-based interpolation module, which improves previous CDA as follows: (1) it decreases the number of false-detections for the required number of true detections; (2) it improves detection capabilities for very small craters; and (3) it provides more accurate automated measurements of craters' properties. The results are: (1) LU60645GT, which is currently the most complete (up to ∼D>=8 km) catalogue of Lunar craters; and (2) MA132843GT catalogue of Martian craters complete up to ∼D>=2 km, which is the extension of the previous MA130301GT catalogue. As previously achieved for Mars, LU60645GT provides all properties that were provided by the previous Lunar catalogues, plus: (1) correlation between morphological descriptors from used catalogues; (2) correlation between manually assigned attributes and automated measurements; (3) average errors and their standard deviations for manually and automatically assigned attributes such as position coordinates, diameter, depth/diameter ratio, etc; and (4) a review of positional accuracy of used datasets. Additionally, surface dating could potentially be improved with the exhaustiveness of this new catalogue. The accompanying results are: (1) the possibility of comparing a large number of Lunar and Martian craters, of e.g. depth/diameter ratio and 2D profiles; (2) utilisation of a method for re-projection of datasets and catalogues, which is very useful for craters that are very close to poles; and (3) the extension of the previous framework for evaluation of CDAs with datasets and ground-truth catalogue for the Moon.

Craters↗
NASA NTRS2014

Xilinx Virtex-5QV (V5QV) Independent SEU Data

This is an independent study to determine the single event destructive and transient susceptibility of the Xilinx Virtex-5QV (SIRF) device. A framework for evaluating complex digital systems targeted for harsh radiation environments such as space is presented.

Field Programmable Gate Array (FPGA)↗